Sharing television and video programming through social networking

ABSTRACT

In particular embodiments, one or more computer systems of a social-networking system query a social graph of the social-networking system for social content associated with particular media content. The social graph includes nodes and edges connecting the nodes. The nodes include user nodes that are each associated with a particular user of the social-networking system. The one or more computer systems of the social-networking system determine a time associated with each of the social content, determine, using the determined time of each of the social content, a particular point within the particular media content that the social content was generated, and present a content visualization graph for display on a display device. The content visualization graph is a visual indication of the particular points within the particular media content that the social content was generated.

RELATED APPLICATION

The present application is a divisional application of U.S. patentapplication Ser. No. 13/602,011, filed Aug. 31, 2012, and entitled“Sharing Television and Video Programming Through Social Networking”

TECHNICAL FIELD

This disclosure relates generally to social networking.

BACKGROUND

A social-networking system, which may include a social-networkingwebsite, may enable its users (such as persons or organizations) tointeract with it and with each other through it. The social-networkingsystem may, with input from a user, create and store in thesocial-networking system a user profile associated with the user. Theuser profile may include demographic information, communication-channelinformation, and information on personal interests of the user. Thesocial-networking system may also, with input from a user, create andstore a record of relationships of the user with other users of thesocial-networking system, as well as provide services (e.g. wall posts,photo-sharing, event organization, messaging, games, or advertisements)to facilitate social interaction between or among users.

The social-networking system may send over one or more networks contentor messages related to its services to a mobile or other computingdevice of a user. A user may also install software applications on amobile or other computing device of the user for accessing a userprofile of the user and other data within the social-networking system.The social-networking system may generate a personalized set of contentobjects to display to a user, such as a newsfeed of aggregated storiesof other users connected to the user.

A mobile computing device—such as a smartphone, tablet computer, orlaptop computer—may include functionality for determining its location,direction, or orientation, such as a GPS receiver, compass, orgyroscope. Such a device may also include functionality for wirelesscommunication, such as BLUETOOTH communication, near-field communication(NFC), or infrared (IR) communication or communication with wirelesslocal area networks (WLANs) or cellular-telephone network. Such a devicemay also include one or more cameras, scanners, touchscreens,microphones, or speakers. Mobile computing devices may also executesoftware applications, such as games, web browsers, or social-networkingapplications. With social-networking applications, users may connect,communicate, and share information with other users in their socialnetworks.

SUMMARY OF PARTICULAR EMBODIMENTS

Particular embodiments enhance a user's television viewing experience byproviding for display to the user a comment visualization graph for aparticular program. The comment visualization graph presents a visualrepresentation of the amount of comments made during the program and atwhat point during the program they were made.

In certain embodiments, comments, posts, chat messages, or any othercontent posted to a social networking system may be time-stamped withinformation indicating when the content was posted. For example, a postabout a particular TV show may be time-stamped with a time thatindicates when in the show the post was made. As another example, chatmessages about a particular movie may be time-stamped to indicate atwhat point in the movie the chat messages were exchanged. In someembodiments, the social networking system analyzes content posted to thesocial networking system and determines all posts related to aparticular program. A comment visualization graph may then be createdand displayed to the user on, for example, a mobile device or a TV.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates an example network environment associated with asocial-networking system.

FIG. 2 illustrates an example social-networking system.

FIG. 3 illustrates an example social graph.

FIG. 4 illustrates an example network environment of an examplesocial-network environment.

FIG. 5 illustrates an example cycle of sharing of watched video content.

FIG. 6 illustrates example sharing of a story regarding a content objectto multiple external services.

FIG. 7 illustrates an example method for providing instructions to aset-top box (STB) of a user.

FIG. 8 illustrates an example network environment of an example socialTV dongle.

FIG. 9 illustrates an example social TV dongle.

FIG. 10 illustrates an example method for concurrently displaying socialcontent with content being viewed on a TV.

FIG. 11 illustrates an example modified video stream.

FIG. 12 illustrates an example method for displaying social content ontop of content currently being viewed on a TV.

FIG. 13 illustrates an example environment for determining what a useris watching.

FIG. 14 illustrates an example method for determining what a user iscurrently watching.

FIG. 15 illustrates an example environment for determining identities ofone or more users who are watching content on a TV.

FIG. 16 illustrates an example method for determining identities of oneor more users who are watching content on a TV.

FIG. 17 illustrates an example environment for determining one or moreuser products.

FIG. 18 illustrates an example method for determining one or more userproducts.

FIG. 19 illustrates an example environment for determining productsplaced in programs.

FIG. 20 illustrates an example method for determining products placed inprograms.

FIG. 21 illustrates an example environment in which a second screen isutilized to present information related to what is currently beingdisplayed on a TV.

FIG. 22 illustrates an example method for utilizing a second screen topresent information related to what is currently being displayed on aTV.

FIG. 23 illustrates an example environment for utilizing a second screento participate in a chat session related to what is currently beingdisplayed on a TV.

FIG. 24 illustrates an example method for utilizing a second screen toparticipate in a chat session related to what is currently beingdisplayed on a TV.

FIG. 25 illustrates an example playlist for content to be viewed on aTV.

FIG. 26 illustrates an example method of generating an example sharedplaylist.

FIG. 27 illustrates example time-sensitive buttons that may be displayedin response to what is being shown on a TV.

FIG. 28 illustrates an example method for displaying time-sensitivebuttons in response to what is being shown on a TV.

FIG. 29 illustrates an example environment in which devices may beutilized as smart remote controls for a TV.

FIG. 30 illustrates an example method for utilizing devices as smartremote controls for a TV.

FIG. 31 illustrates an example social programming guide (SPG).

FIG. 32 illustrates an example method for providing an SPG.

FIG. 33 illustrates an example social graph.

FIG. 34 illustrates an example method for streaming media to multipleuser devices.

FIG. 35 illustrates an example comment visualization graph.

FIG. 36 illustrates an example method for providing an example commentvisualization graph.

FIGS. 37A-B illustrate an example personal computing device.

FIG. 38 illustrates an example software architecture for information andapplications on a personal computing device.

FIG. 39 illustrates an example computing system.

DESCRIPTION OF EXAMPLE EMBODIMENTS

Particular embodiments enhance a user's television viewing experience byproviding for display to the user a comment visualization graph for aparticular program. The comment visualization graph presents a visualrepresentation of the amount of comments made during the program and atwhat point during the program they were made. In certain embodiments,comments, posts, chat messages, or any other content posted to a socialnetworking system may be time-stamped with information indicating whenthe content was posted. For example, a post about a particular TV showmay be time-stamped with a time that indicates when in the show the postwas made. As another example, chat messages about a particular movie maybe time-stamped to indicate at what point in the movie the chat messageswere exchanged. In some embodiments, the social networking systemanalyzes content posted to the social networking system and determinesall posts related to a particular program. A comment visualization graphmay then be created and displayed to the user on, for example, a mobiledevice or a TV.

FIG. 1 illustrates an example network environment 100 associated with asocial-networking system. Network environment 100 includes a user 101, aclient system 130, a social-networking system 160, and a third-partysystem 170 connected to each other by a network 110. Although FIG. 1illustrates a particular arrangement of user 101, client system 130,social-networking system 160, third-party system 170, and network 110,this disclosure contemplates any suitable arrangement of user 101,client system 130, social-networking system 160, third-party system 170,and network 110. As an example and not by way of limitation, two or moreof client system 130, social-networking system 160, and third-partysystem 170 may be connected to each other directly, bypassing network110. As another example, two or more of client system 130,social-networking system 160, and third-party system 170 may bephysically or logically co-located with each other in whole or in part.Moreover, although FIG. 1 illustrates a particular number of users 101,client systems 130, social-networking systems 160, third-party systems170, and networks 110, this disclosure contemplates any suitable numberof users 101, client systems 130, social-networking systems 160,third-party systems 170, and networks 110. As an example and not by wayof limitation, network environment 100 may include multiple users 101,client system 130, social-networking systems 160, third-party systems170, and networks 110.

In particular embodiments, user 101 may be an individual (human user),an entity (e.g., an enterprise, business, or third-party application),or a group (e.g., of individuals or entities) that interacts orcommunicates with or over social-networking system 160. In particularembodiments, social-networking system 160 may be a network-addressablecomputing system hosting an online social network. Social-networkingsystem 160 may generate, store, receive, and send social-networkingdata, such as, for example, user-profile data, concept-profile data,social-graph information, or other suitable data related to the onlinesocial network. Social-networking system 160 may be accessed by theother components of network environment 100 either directly or vianetwork 110. In particular embodiments, social-networking system 160 mayinclude an authorization server that allows users 101 to opt in or optout of having their actions logged by social-networking system 160 orshared with other systems (e.g., third-party systems 170), such as, forexample, by setting appropriate privacy settings. In particularembodiments, third-party system 170 may be a network-addressablecomputing system that can host websites that embed data fromsocial-networking system 160. Third-party system 170 may be accessed bythe other components of network environment 100 either directly or vianetwork 110. In particular embodiments, one or more users 101 may useone or more client systems 130 to access, send data to, and receive datafrom social-networking system 160 or third-party system 170. Clientsystem 130 may access social-networking system 160 or third-party system170 directly, via network 110, or via a third-party system. As anexample and not by way of limitation, client system 130 may accessthird-party system 170 via social-networking system 160. Client system130 may be any suitable computing device, such as, for example, apersonal computer, a laptop computer, a cellular telephone, asmartphone, or a tablet computer.

This disclosure contemplates any suitable network 110. As an example andnot by way of limitation, one or more portions of network 110 mayinclude an ad hoc network, an intranet, an extranet, a virtual privatenetwork (VPN), a local area network (LAN), a wireless LAN (WLAN), a widearea network (WAN), a wireless WAN (WWAN), a metropolitan area network(MAN), a portion of the Internet, a portion of the Public SwitchedTelephone Network (PSTN), a cellular telephone network, or a combinationof two or more of these. Network 110 may include one or more networks110.

Links 150 may connect client system 130, social-networking system 160,and third-party system 170 to communication network 110 or to eachother. This disclosure contemplates any suitable links 150. Inparticular embodiments, one or more links 150 include one or morewireline (such as for example Digital Subscriber Line (DSL) or Data overCable Service Interface Specification (DOCSIS)), wireless (such as forexample Wi-Fi or Worldwide Interoperability for Microwave Access(WiMAX)), or optical (such as for example Synchronous Optical Network(SONET) or Synchronous Digital Hierarchy (SDH)) links. In particularembodiments, one or more links 150 each include an ad hoc network, anintranet, an extranet, a VPN, a LAN, a WLAN, a WAN, a WWAN, a MAN, aportion of the Internet, a portion of the PSTN, a cellulartechnology-based network, a satellite communications technology-basednetwork, another link 150, or a combination of two or more such links150. Links 150 need not necessarily be the same throughout networkenvironment 100. One or more first links 150 may differ in one or morerespects from one or more second links 150.

FIG. 2 is a diagram of one embodiment of a social networking system 160.The embodiment of a social networking system 160 shown by FIG. 2includes a web server 210, an action logger 215, an API request server220, a relevance and ranking engine 225, a content object classifier260, a notification controller 265, an action log 230, a third-partycontent object exposure log 270, an inference module 275, anauthorization server 235, a search module 280, an ad targeting module285, a user interface module 290, a user profile store 240, a connectionstore 245, a third-party content store 250, and a location store 255. Inother embodiments, the social networking system 160 may includeadditional, fewer, or different modules for various applications.Conventional components such as network interfaces, security mechanisms,load balancers, failover servers, management and network operationsconsoles, and the like are not shown so as to not obscure the details ofthe system.

As described above in conjunction with FIG. 1, the social networkingsystem 160 comprises a computing system that allows users to communicateor otherwise interact with each other and access content as describedherein. The social networking system 160 stores user profiles describingthe users of a social network in a user profile store 240. The userprofiles include biographic, demographic, and other types of descriptiveinformation, such as work experience, educational history, hobbies orpreferences, interests, location, and the like. For example, the userprofile store 240 contains data structures with fields suitable fordescribing a user's profile. When a new object of a particular type iscreated, the social networking system 160 initializes a new datastructure, i.e., a “node” of the corresponding type, assigns a uniqueobject identifier to it, and begins to add data to the object as needed.This might occur, for example, when a user becomes a user of the socialnetworking system 160, the social networking system 160 generates a newinstance of a user profile in the user profile store 240, assigns aunique identifier to the user profile, and begins to populate the fieldsof the user profile with information provided by the user.

In addition, the user profile store 240 may include data structuressuitable for describing a user's demographic data, behavioral data, andother social data. Demographic data typically includes data about theuser, such as age, gender, location, etc., e.g., as included in theuser's profile. Behavioral data typically includes information about theuser's activities within the social networking system 160, such asspecific actions (posts, likes, comments, etc.), activity levels, usagestatistics, etc. Other social data comprises information about the userfrom within the social networking system 160 that is not strictlydemographic or behavioral, such as interests or affinities, etc. In oneembodiment, user's interests may be explicitly specified in the user'sprofile or interests that may be inferred from the user's activities inthe social networking system (e.g., uploaded content, postings, readingof messages, etc). Additionally, the user profile store 240 includeslogic for maintaining user interest information for users according toone or more categories. Categories may be general or specific, e.g., ifa user “likes” an article about a brand of shoes the category may be thebrand, or the general category of “shoes” or “clothing.” Multiplecategories may apply to a single user interest. In addition, the userprofile store 240 may be accessed by other aspects of the socialnetworking system 160.

For example, the user profile store 240 includes logic for maintaininginterest information for users according to one or more categories.Categories may be general or specific, e.g., if a user “likes” anarticle about a brand of shoes the category may be the brand, or thegeneral category of “shoes” or “clothing.” Multiple categories may applyto a single user interest. In addition, the user profile store 240 maybe accessed by other aspects of the social networking system 160.

The social networking system 160 further stores data describing one ormore connections between different users in a user connection store 245.The connection information may indicate users who have similar or commonwork experience, group memberships, hobbies, educational history, or arein any way related or share common attributes. Additionally, the socialnetworking system 160 includes user-defined connections betweendifferent users, allowing users to specify their relationships withother users. For example, user-defined connections allow users togenerate relationships with other users that parallel the users'real-life relationships, such as friends, co-workers, partners, and soforth. Users may select from predefined types of connections, or definetheir own connection types as needed. The connection store 245 includesdata structures suitable for describing a user's connections to otherusers, connections to third-party content object providers 170, orconnections to other entities. The connection stores 245 may alsoassociate a connection type with a user's connections, which may be usedin conjunction with the user's privacy setting, to regulate access toinformation about the user. In addition, the connection store 245 may beaccessed by other aspects of the social networking system 160.

The web server 210 links the social networking system to one or moreclient devices 130 and/or one or more third-party content objectproviders 170 via the network 110. The web server 210 serves web pages,as well as other web-related content, such as Java, Flash, XML, and soforth. The web server 210 may include a mail server or other messagingfunctionality for receiving and routing messages between the socialnetworking system 160 and one or more client devices 130. The messagescan be instant messages, queued messages (e.g., email), text and SMSmessages, or any other suitable messaging format.

The Application Programming Interface (API) request server 220 allowsone or more third-party content object providers 170 to accessinformation from the social networking system 160 by calling one or moreAPIs. The API request server 220 also may allow third-party contentobject providers 170 to send information to the social networking systemby calling APIs. For example, a third-party content object provider 170sends an API request to the social networking system 160 via the network110 and the API request server 220 receives the API request. The APIrequest server 220 processes the request by calling an API associatedwith the API request to generate an appropriate response, which the APIrequest server 220 communicates to the third-party content objectprovider 170 via the network 110.

The action logger 215 is capable of receiving communications from theweb server 210 about user actions on and/or off the social networkingsystem 160. The action logger 215 populates the action log 230 withinformation about user actions, allowing the social networking system160 to track or monitor various actions taken by its users within thesocial networking system 160 and outside of the social networking system160. Any action that a particular user takes with respect to anotheruser is associated with each user's profile, through informationmaintained in the action log 230 or in a similar database or other datarepository. Examples of actions taken by a user within the socialnetwork 160 that are identified and stored may include, for example,adding a connection to another user, sending a message to another user,reading a message from another user, viewing content associated withanother user, attending an event posted by another user or other actionsinteracting with another user. When a user takes an action within thesocial networking system 160, the action is recorded in the action log230. In one embodiment, the social networking system maintains theaction log 230 as a database of entries. When an action is taken withinthe social networking system 160, an entry for the action is added tothe action log 230. The relevance and ranking engine 225 includes logicfor calculating a relevance score for content objects (including bothuser-generated content objects and third-party content objects) relativeto a user, for ranking the content objects by their relevance scores,and for selecting content objects for sending to users as notificationsor as responses to user requests. To calculate the relevance score, therelevance and ranking engine 225 determines a location value bycomparing the content object location and a current location for theuser device 210, determines an interest value based on whether thecontent object categories are included in the user's interests,determines a time value based on whether the current time is within thedelivery time range for the content object, and determines a connectionvalue based on how many of the user's connections are associated withthe content object. Then, the relevance and ranking engine 225 combinesthe location value, interest value, connection value, and time value todetermine the relevance score for the content object with respect to theuser. In one embodiment the values are higher for a better fit (closerproximity, great similarity, etc.) and approach a value of one, and aremultiplied together to yield the relevance score. From the relevancescores for each content object, the relevance and ranking engine 225ranks the content objects for a user, e.g., from highest relevance scoreto lowest. The relevance and ranking engine 225 then can select contentobjects to send to a notification controller 265, or can serve thehighest ranked content object directly to the client device 130 as anotification(s).

The content object classifier 260 includes logic for assigning each ofthe content objects a location, a category, and a delivery time range.Categories may reflect various categories of user interests, and may beassociated with the interests themselves, e.g., a user “likes” anarticle about a brand of shoes and the category is the brand, or thearticle about the shoe brand is assigned a general category of “shoes”or “clothing.” Multiple categories may apply to a single content object.General or specific locations may be assigned to content objects aswell, e.g., a city, a particular street name or intersection, or GPScoordinates. A delivery time range is assigned to each content object,e.g., using a useful range based on the hours the associated business isopen.

Additionally, user actions may be associated with exposure tothird-party content objects from one or more third-party content objectproviders 170. Thus, in conjunction with the action log 230, athird-party content object log 270 is maintained of user exposures tosuch objects and when the last exposure occurred. The action logger 215receives data describing a user's interaction with an object and storesit to the third-party content object log 270. The third-party contentobject log 270 includes logic for storing user exposures to third-partycontent objects and associations between users and objects. The exposureinformation can be used to determine whether to expose the user to thesame or similar content objects, and for adjusting the ranking andselection of content objects on the basis of whether the user previouslyhas been exposed to the same or similar content object. In addition, ifa user becomes associated with a content object via an action, e.g.,uses an incentive, goes to the location, etc., that information also isstored, and can be used for re-ranking and re-selecting the contentobjects.

The notification controller 265 provides information regarding contentobjects to the client device 130. Information may be pushed to theclient device 130 as notifications, or information may be pulled to theuser device responsive to a request received from the client device 130.In the push circumstance, notifications of content objects are initiallypushed according to a default rate. Based on user engagement with thenotifications, the notification controller 265 may adjust the rate inwhich notifications are provided to the client device 130. By adjustingthe initial settings, the notification controller 265 providesnotifications of content objects to the client device 130 when the useris more likely to engage with the notifications. Information may bepulled to the user device at any time. Additionally, the type of contentthat is provided to the client device 130 may be updated based on theuser engagement.

The authorization server 235 enforces one or more privacy settings ofthe users of the social networking system 160. A privacy setting of auser determines how particular information associated with a user can beshared. The privacy setting comprises the specification of particularinformation associated with a user and the specification of the entityor entities with whom the information can be shared. Examples ofentities with which information can be shared may include other users,applications, external websites or any entity that can potentiallyaccess the information. The information that can be shared by a usercomprises user profile information like profile photo, phone numbersassociated with the user, user's connections, actions taken by the usersuch as adding a connection, changing user profile information and thelike.

The useful social information that is tracked and maintained by a socialnetworking system can be thought of in terms of a “social graph,” whichincludes a plurality of nodes that are interconnected by a plurality ofedges. Each node in the social graph may represent something that canact on and/or be acted upon by another node. Common examples of nodesinclude users, non-person entities, content objects, groups, events,messages, concepts, and any other things that can be represented by anobject in a social networking system. An edge between two nodes in thesocial graph represents a particular kind of connection between the twonodes, which may result from an action that was performed by one of thenodes on the other node.

The social networking system 160 may receive a request to associate theweb content with a node in the social networking system 160. An externalwebsite (e.g., of the third party content object provider 170)incorporates a tag into the markup language document for the web page(s)of the web content to claim ownership of the pages/domain in the contextof the social networking system 160. In some cases, an entire domain orcollection of web pages is associated with a unique identifier thatassociates the web pages with a node. Once established, the socialnetworking system 160 tracks data associated with the node in the actionlog 230.

Data stored in the connection store 245, the user profile store 240 andthe action log 230 allows the social networking system 160 to generate asocial graph that uses nodes to identify various objects and edgesconnecting nodes to identify relationships between different objects. Anedge between two nodes in the social graph represents a particular kindof connection between the two nodes, which may result from an actionthat was performed by one of the nodes on the other node.

The third-party content object store 250 stores content objects receivedfrom third parties. The third-party content objects includeinformational content objects, such as movie show times, restaurantmenus, etc., as well as incentive content objects, such as coupons,discount tickets, gift certificates, etc. In addition, some third-partycontent objects may include a combination of information and incentives.

The location store 255 stores location information received from clientdevices 130 associated with users. The location information used by thesocial networking system 160 may be obtained directly from clientdevices 130, e.g., at the time a notification is to be sent or atvarious predetermined time intervals, or the location information may bea last stored location received from the client device 130. In addition,the location store 255 may receive updated location information, e.g.,in response to a change in the location of a client device 130. In oneembodiment, if an updated location is received, the updated location isprovided to the relevance and ranking engine 225 for re-ranking and/orre-selecting the third-party content objects in view of the updatedlocation information.

In general, the selection or ranking of content objects may occur atvarying intervals based on several variables, such as always at thebeginning of a period during which a notification would be served, orevery X minutes during a period during which notifications will beserved, or every X minutes all the time (e.g., so that it's ready when asearch happens), only in response to a change in location or expirationof a delivery time for a content object, etc. Alternatively, the rankingof content objects may occur as a result of user demand. The user mayexplicitly request the ranking by submitting a request for relevantinformation happening within the vicinity of the user. The request maybe received in response to user selection of a “refresh” elementincluded in a user application associated with the present disclosure.The request may also be implicit. For example, upon launching of theuser application, a request may be automatically received for theranking.

The social networking system 160 implements context search using acontext search module 280. Context search results are search resultsthat are relevant to the user based on their current location as well astheir social information. In this way, the context search results aretailored to the user's interests, connections, and location at the timeof the search. The context search module 280 incorporates locationinformation, search results and relevance score information obtainedfrom the relevance and ranking engine 225 in order to provide a rankedlist of search results and/or for selection of third-party contentobjects as the basis for serving notifications.

The ad pricing module 285 combines social information, the current time,and location information to provide relevant advertisements, in the formof notifications, to a user. Advertisements of increased relevance to auser are more likely to result in a purchase. Dividing consumersaccording to their interests based on social information allowsmerchants to calculate the value of their potential customers.Advertisements provided through the social networking system 160 may bepriced according to the value of the customer to the merchant, asindicated by their social information.

In one embodiment, the UI (or User Interface) module 290 is configuredto display a map containing pins on a client device 130, where each pinrepresents a content object in actionable proximity to a user (e.g., anearby friend, deal, etc.). The content objects selected to be presentedas pins to the user are those content objects with sufficiently highrelevance scores. Multiple pins for a nearby area can be clustered. TheUI module 290 provides the user with the ability to switch between zoomlevels for the map, thereby showing pins at varying distances from theuser's current location (e.g., nearby, close, and far). In oneembodiment, the zoom levels are based on the existence of relevantcontent rather than being predetermined distances from the user.

In one embodiment, the UI module 290 is configured to display a rankedlist of search results on a client device 130 that have been ranked bythe context search module 280. The UI module 290 is additionallyconfigured to generate an advertisement dashboard for merchantsadvertising through the social networking system 160. The advertisementdashboard allows merchants to control the distribution and price theypay for their advertisements. For both functions, the UI module isconfigured to generate a user interface that a client device 130 or athird-party content object provider (or merchant) 130 may interact with.

An inference module 275 determines overlapping interests between usersin the social networking system 160. By determining the overlappinginterests between a user and his or her friends, the inference module275 may identify which interests may be imputed to the user based on theinterests of the user's friends. Thus, through the user's friends, theinference module 275 allows the social networking system 160 to identifyinterests for the user that are not explicitly indicated by the user.

The third-party content object store 250 stores content objects receivedfrom third parties. The third-party content objects includeinformational content objects, such as movie show times, movie reviews,restaurant reviews, restaurant menus, product information and reviews,etc., as well as incentive content objects, such as coupons, discounttickets, gift certificates, etc. In addition, some third-party contentobjects may include a combination of information and incentives.

The location store 255 stores location information received from userdevices associated with users. The location information used by thesocial networking system 160 may be obtained directly from clientdevices 130, e.g., at the time a notification is to be sent or atvarious predetermined time intervals, or the location information may bea last stored location received from the client device 130. The locationinformation may also be obtained along with a request from a user. Inaddition, the location store 255 may receive updated locationinformation, e.g., in response to a change in the location of a clientdevice 130. In one embodiment, if an updated location is received, theupdated location is provided to the relevance and ranking engine 225 forre-ranking and or re-selection of the content objects in view of theupdated location information.

FIG. 3 illustrates an example social graph 300. In particularembodiments, social-networking system 160 may store one or more socialgraphs 300 in one or more data stores. In particular embodiments, socialgraph 300 may include multiple nodes—which may include multiple usernodes 302 or multiple concept nodes 304—and multiple edges 306connecting the nodes. Example social graph 300 illustrated in FIG. 3 isshown, for didactic purposes, in a two-dimensional visual maprepresentation. In particular embodiments, a social-networking system160, client system 130, or third-party system 170 may access socialgraph 300 and related social-graph information for suitableapplications. The nodes and edges of social graph 300 may be stored asdata objects, for example, in a data store (such as a social-graphdatabase). Such a data store may include one or more searchable orqueryable indexes of nodes or edges of social graph 300.

In particular embodiments, a user node 302 may correspond to a user ofsocial-networking system 160. As an example and not by way oflimitation, a user may be an individual (human user), an entity (e.g.,an enterprise, business, or third-party application), or a group (e.g.,of individuals or entities) that interacts or communicates with or oversocial-networking system 160. In particular embodiments, when a userregisters for an account with social-networking system 160,social-networking system 160 may create a user node 302 corresponding tothe user, and store the user node 302 in one or more data stores. Usersand user nodes 302 described herein may, where appropriate, refer toregistered users and user nodes 302 associated with registered users. Inaddition or as an alternative, users and user nodes 302 described hereinmay, where appropriate, refer to users that have not registered withsocial-networking system 160. In particular embodiments, a user node 302may be associated with information provided by a user or informationgathered by various systems, including social-networking system 160. Asan example and not by way of limitation, a user may provide his or hername, profile picture, contact information, birth date, sex, maritalstatus, family status, employment, education background, preferences,interests, or other demographic information. In particular embodiments,a user node 302 may be associated with one or more data objectscorresponding to information associated with a user. In particularembodiments, a user node 302 may correspond to one or more webpages orone or more user-profile pages (which may be webpages).

In particular embodiments, a concept node 304 may correspond to aconcept. As an example and not by way of limitation, a concept maycorrespond to a place (such as, for example, a movie theater,restaurant, landmark, or city); a website (such as, for example, awebsite associated with social-network system 160 or a third-partywebsite associated with a web-application server); an entity (such as,for example, a person, business, group, sports team, or celebrity); aresource (such as, for example, an audio file, video file, digitalphoto, text file, structured document, or application) which may belocated within social-networking system 160 or on an external server,such as a web-application server; real or intellectual property (suchas, for example, a sculpture, painting, movie, game, song, idea,photograph, or written work); a game; an activity; an idea or theory;another suitable concept; or two or more such concepts. A concept node304 may be associated with information of a concept provided by a useror information gathered by various systems, including social-networkingsystem 160. As an example and not by way of limitation, information of aconcept may include a name or a title; one or more images (e.g., animage of the cover page of a book); a location (e.g., an address or ageographical location); a website (which may be associated with a URL);contact information (e.g., a phone number or an email address); othersuitable concept information; or any suitable combination of suchinformation. In particular embodiments, a concept node 304 may beassociated with one or more data objects corresponding to informationassociated with concept node 304. In particular embodiments, a conceptnode 304 may correspond to a webpage.

In particular embodiments, a node in social graph 300 may represent orbe represented by a webpage (which may be referred to as a “profilepage”). Profile pages may be hosted by or accessible tosocial-networking system 160. Profile pages may also be hosted onthird-party websites associated with a third-party server 170. As anexample and not by way of limitation, a profile page corresponding to aparticular external webpage may be the particular external webpage andthe profile page may correspond to a particular concept node 304.Profile pages may be viewable by all or a selected subset of otherusers. As an example and not by way of limitation, a user node 302 mayhave a corresponding user-profile page in which the corresponding usermay add content, make declarations, or otherwise express himself orherself. As another example and not by way of limitation, a concept node304 may have a corresponding concept-profile page in which one or moreusers may add content, make declarations, or express themselves,particularly in relation to the concept corresponding to concept node304.

In particular embodiments, a concept node 304 may represent athird-party webpage or resource hosted by a third-party system 170. Thethird-party webpage or resource may include, among other elements,content, a selectable or other icon, or other inter-actable object(which may be implemented, for example, in JavaScript, AJAX, or PHPcodes) representing an action or activity. As an example and not by wayof limitation, a third-party webpage may include a selectable icon suchas “like,” “check in,” “eat,” “recommend,” or another suitable action oractivity. A user viewing the third-party webpage may perform an actionby selecting one of the icons (e.g., “eat”), causing a client system 130to send to social-networking system 160 a message indicating the user'saction. In response to the message, social-networking system 160 maycreate an edge (e.g., an “eat” edge) between a user node 302corresponding to the user and a concept node 304 corresponding to thethird-party webpage or resource and store edge 306 in one or more datastores.

In particular embodiments, a pair of nodes in social graph 300 may beconnected to each other by one or more edges 306. An edge 306 connectinga pair of nodes may represent a relationship between the pair of nodes.In particular embodiments, an edge 306 may include or represent one ormore data objects or attributes corresponding to the relationshipbetween a pair of nodes. As an example and not by way of limitation, afirst user may indicate that a second user is a “friend” of the firstuser. In response to this indication, social-networking system 160 maysend a “friend request” to the second user. If the second user confirmsthe “friend request,” social-networking system 160 may create an edge306 connecting the first user's user node 302 to the second user's usernode 302 in social graph 300 and store edge 306 as social-graphinformation in one or more of data stores 24. In the example of FIG. 3,social graph 300 includes an edge 306 indicating a friend relationbetween user nodes 302 of user “A” and user “B” and an edge indicating afriend relation between user nodes 302 of user “C” and user “B.”Although this disclosure describes or illustrates particular edges 306with particular attributes connecting particular user nodes 302, thisdisclosure contemplates any suitable edges 306 with any suitableattributes connecting user nodes 302. As an example and not by way oflimitation, an edge 306 may represent a friendship, family relationship,business or employment relationship, fan relationship, followerrelationship, visitor relationship, subscriber relationship,superior/subordinate relationship, reciprocal relationship,non-reciprocal relationship, another suitable type of relationship, ortwo or more such relationships. Moreover, although this disclosuregenerally describes nodes as being connected, this disclosure alsodescribes users or concepts as being connected. Herein, references tousers or concepts being connected may, where appropriate, refer to thenodes corresponding to those users or concepts being connected in socialgraph 300 by one or more edges 306.

In particular embodiments, an edge 306 between a user node 302 and aconcept node 304 may represent a particular action or activity performedby a user associated with user node 302 toward a concept associated witha concept node 304. As an example and not by way of limitation, asillustrated in FIG. 3, a user may “like,” “attended,” “played,”“listened,” “cooked,” “worked at,” or “watched” a concept, each of whichmay correspond to a edge type or subtype. A concept-profile pagecorresponding to a concept node 304 may include, for example, aselectable “check in” icon (such as, for example, a clickable “check in”icon) or a selectable “add to favorites” icon. Similarly, after a userclicks these icons, social-networking system 160 may create a “favorite”edge or a “check in” edge in response to a user's action correspondingto a respective action. As another example and not by way of limitation,a user (user “C”) may listen to a particular song (“Imagine”) using aparticular application (SPOTIFY, which is an online music application).In this case, social-networking system 160 may create a “listened” edge306 and a “used” edge (as illustrated in FIG. 3) between user nodes 302corresponding to the user and concept nodes 304 corresponding to thesong and application to indicate that the user listened to the song andused the application. Moreover, social-networking system 160 may createa “played” edge 306 (as illustrated in FIG. 3) between concept nodes 304corresponding to the song and the application to indicate that theparticular song was played by the particular application. In this case,“played” edge 306 corresponds to an action performed by an externalapplication (SPOTIFY) on an external audio file (the song “Imagine”).Although this disclosure describes particular edges 306 with particularattributes connecting user nodes 302 and concept nodes 304, thisdisclosure contemplates any suitable edges 306 with any suitableattributes connecting user nodes 302 and concept nodes 304. Moreover,although this disclosure describes edges between a user node 302 and aconcept node 304 representing a single relationship, this disclosurecontemplates edges between a user node 302 and a concept node 304representing one or more relationships. As an example and not by way oflimitation, an edge 306 may represent both that a user likes and hasused at a particular concept. Alternatively, another edge 306 mayrepresent each type of relationship (or multiples of a singlerelationship) between a user node 302 and a concept node 304 (asillustrated in FIG. 3 between user node 302 for user “E” and conceptnode 304 for “SPOTIFY”).

In particular embodiments, social-networking system 160 may create anedge 306 between a user node 302 and a concept node 304 in social graph300. As an example and not by way of limitation, a user viewing aconcept-profile page (such as, for example, by using a web browser or aspecial-purpose application hosted by the user's client system 130) mayindicate that he or she likes the concept represented by the conceptnode 304 by clicking or selecting a “Like” icon, which may cause theuser's client system 130 to send to social-networking system 160 amessage indicating the user's liking of the concept associated with theconcept-profile page. In response to the message, social-networkingsystem 160 may create an edge 306 between user node 302 associated withthe user and concept node 304, as illustrated by “like” edge 306 betweenthe user and concept node 304. In particular embodiments,social-networking system 160 may store an edge 306 in one or more datastores. In particular embodiments, an edge 306 may be automaticallyformed by social-networking system 160 in response to a particular useraction. As an example and not by way of limitation, if a first useruploads a picture, watches a movie, or listens to a song, an edge 306may be formed between user node 302 corresponding to the first user andconcept nodes 304 corresponding to those concepts. Although thisdisclosure describes forming particular edges 306 in particular manners,this disclosure contemplates forming any suitable edges 306 in anysuitable manner.

FIG. 4 illustrates an example network environment for an example socialnetworking system enabling social TV viewing. Social networking system160 may be connected via network 110 such as the Internet to a pluralityof connected devices 404. Connected devices 404 may include, withoutlimitation, Internet-enabled television sets 404 a that connect tonetwork 110 through a local internet service provider (ISP), mobiledevices 404 b that connect to network 110 through a wireless connectionsuch as a wireless cellular data network, or TVs 404 d that connect tothe Internet (network 110) through a STB or gateway device 404 c.STB/gateway 404 c may be any hardware or software that delivers contentor possesses a network interface card (NIC) for connecting to a localarea network (LAN). For example, STB/gateway 404 c may be a cable boxprovided by an MSO, such as Comcast, Time Warner, AT&T U-verse, or DishNetwork. In such examples, STB/gateway 404 c may receive content fromMSOs 402. As another example, STB/gateway 404 c may be a device thatstreams video from third-party internet sites or services such as OTTproviders 403. Examples of such gateways 404 c include, withoutlimitation, devices from Roku, Boxee, Apple TV, and Google TV that allowusers to access content from OTT providers 403 such as NetFlix, Hulu,Amazon Video, YouTube, and the like. In some embodiments, STB/gateway404 c may be a stand-alone device. In other embodiments, thefunctionality of STB/gateway 404 c may be incorporated into TVs 404 d.

In general, a user's social connections or activities can be used tocustomize or personalize the user's experience with a social TV viewingsystem such as the system of FIG. 4. Some embodiments may utilizemethods to customize a user's experience as described in U.S. patentapplication Ser. No. 12/759,676 entitled “Token-Activated, FederatedAccess to Social Network Information,” which is incorporated herein byreference. In particular embodiments, connected devices 404 receivecontent from MSOs 402 or OTT providers 403 while receiving or sendingsocial data to social networking system 160. For example, a userwatching a particular TV show or movie on either service may choose toshare that he or she is currently viewing the program to his or herfriends. Conversely, a user browsing the Electronic Programming Guide(EPG) of an MSO 402 or the content navigator of an OTT provider 403 maybe presented with social data including the content that his or herfriends on social networking system 160 have watched, are currentlywatching, or plan to watch. Thus, after exchanging auth messages 406with social networking system 160 to authenticate the user's socialnetworking identity with his or her connected device 404, connecteddevices 404 may receive discover messages 408 that identify contentbeing consumed or watched by his or her friends on the social network,and send share messages 407 to inform social networking system 160 ofwhat the user has, is, or plans to watch or “consume.” Although FIG. 4depicts these messages as being sent directly between social networkingsystem 160 and connected devices 404, in particular embodiments,authorization, share, and discover messages 406, 407, and 408 may beexchanged between the provider from which content is being accessed,namely, MSOs 402 and OTT providers 403. This disclosure contemplates anysuitable means of routing messages from viewing devices 404 to socialnetworking system 160.

Although increasingly rare, there are scenarios wherein a particularuser of the social networking system has no means of linking his TV orviewing device to social networking system 160. Unconnected devices 405lack any means of connecting to Internet/network 110. For example, auser may not have a local ISP, and only TV service from an MSO. Asanother example, a user may have both cable service from an MSO as wellas internet access from a local ISP, but his or her STB 404 c may notinclude a NIC. In such configurations, MSO 402 communicatesauthorization, share, and discover messages with social networkingsystem 160, and unconnected devices 405 communicate, via STB 404 c, onlywith MSOs 402.

In particular embodiments, content may be delivered to user devices 404and 405 tagged with content identifiers and metadata. For example,Comcast may utilize its own proprietary EPG data format that lists theprogram name, air date, actors, producer, director, etc. In particularembodiments, content providers 402 and 403 may obtain contentidentifiers and metadata from content databases 401, such as the RoviCorporation. Each particular piece of content may be sent from devices404, or, in particular embodiments, via content providers 402 and 403,as graph data including a graph object and graph action. As previouslydiscussed, social networking system 160 may de-duplicate graph data forthe same graph object in a graph data store by comparing variousattributes about the content object; for example, name, actors,duration, air date, etc. Thus, social networking system 160 mayattribute graph data sent from Hulu that a particular user watched thecritically-acclaimed feature film “The Marine” to the same graph objectas a user currently watching “The Marine” on Comcast digital cable,irrespective of the source and format of the content metadata.

FIG. 5 conceptually illustrates a cycle of sharing of watched videocontent. At authorization phase 501, a particular user 101 authenticatesher social networking user identifier with a device 404 or 405 on whichshe wishes to browse or view content. As previously discussed, devices404/405 may be any device on which a user may view content, including apersonal computing device including a browser application viewing awebsite of an OTT provider 403, television, or mobile device running adedicated application for a content provider. In particular embodiments,a user may associate accounts on multiple external services with his orher social networking identifier, and permit the external services toquery the social graph or publish graph data to social networking system160. For example, user 101 may associate her Hulu account, Netflixaccount, and ATT U-Verse account with her social networking useridentifier and password, thereby sharing all content across multiplecontent delivery services with the social network, and therefore, herfriends on the social network.

At consumption & sharing phase 502, user 101 consumes content bywatching or queuing video content, and shares a story of the consumptionwith social networking system 160. In particular embodiments, user 101may explicitly share a story of her consumption of the content withsocial networking system 160 by actively selecting sharing buttons on auser interface. In particular embodiments, sharing occurs passivelywithout user selection. When stories of content consumption by user 101are sent to social networking system 160, the friends 101 a-e of user101 may view the sharing stories, thereby discovering new content toconsume. In particular embodiments, user 101 may specify which of herfriends 101 a-e may view her consumption stories. In particularembodiments, user 101 may have different privacy settings for activelyand passively shared consumption stories

At discovery phase 503, user 101 may view consumption stories publishedby her friends 101 a-e. As previously stated, the ability of user 101 toview consumption stories published by the user's friends 101 a-e dependson the privacy settings of each individual user 101 a-e. Thus, theconceptual framework of FIG. 5 describes a cycle where users areconstantly discovering new content shared by their friends, consumingcontent, and sharing content with their friends. Hence the userexperience both on and off social networking system 160 is enhanced.

FIG. 6 conceptually illustrates publishing a consumption story acrossmultiple distribution channels. In FIG. 6, user 101 shares a statement605 to social networking system 160 (and consequently, the friends ofuser 101) that she is currently “watching Lone Survivor.” As discussedabove, social networking system 160 processes the graph data receivedfrom the content service provider from which user 101 is watching “LoneSurvivor,” and processes the graph data to map it to a single contentidentifier. The resultant story may be published to a number ofchannels, such as websites 601, applications (“apps”) 602, the socialnetworking system itself 601, and social plug-ins 604.

In particular embodiments, consumption stories are not “pushed” todistribution services 601-604, but rather the service issues a requestto social networking system 160 to query the social graph. For example,when a user logs into the clicker website 601 with their socialnetworking ID, a script of a plug-in embedded in the markup languagedelivered to the user device may issue a query to the social graphcontaining the user's social networking identifier. In response, socialnetworking system 160 may return all the user's friends consumptionstories, and the plug-in may display the consumption stories on apredetermined portion of the page. In particular embodiments, the thirdparty system 170 is responsible for mapping social graph contentidentifiers received from social networking system 160 to content on itsown website.

As another example, when a user accesses a mobile application 602 on hisor her mobile device (such as an iPhone or iPad), the application mayissue a query to the social graph for consumption stories of the friendwhose account is associated with the mobile device. In particularembodiments, social networking system 160 is responsible for the mappingfrom social graph content identifiers to the content identifiers used bythird party system 170. For example, if a user accesses his or herprogramming guide on her AT&T U-Verse service, the social networkingsystem may return data that associates published consumption storieswith programs within the programming guide.

In some embodiments, user 101 may authorize social networking system 160to directly control the user's DVR or STB. For example, a user'sSTB/gateway 404 c may include a DVR for recording TV shows or othercontent for later viewing. User 101 may indicate to social networkingsystem 160 one or more preferences regarding to what extent socialnetworking system 160 is authorized to directly control content that isrecorded on the user's DVR. In one embodiment, the preferences includean authorization for social networking system 160 to directly programthe user's DVR to record TV shows or other content based on what friends101 a-e of user 101 are currently watching or plan to watch. Forexample, social networking system 160 may determine from posts offriends 101 a-e on social networking system 160 that at least one friendis currently watching or planning to watch the show “Lone Survivor.” Anexample of a post may be, “I can't wait to watch Lone Survivor tonight.”If user 101 has authorized social networking system 160 to directlycontrol his DVR, social networking system 160 sends instructions to theuser's DVR to record the show “Lone Survivor.” In certain embodiments,this may include determining a particular start time, channel, andduration for the show “Lone Survivor” and then sending instructions tothe user's DVR to begin recording the determined channel at thedetermined start time and for the determined duration. In someembodiments, the particular start time, channel, and duration of knownshows may be determined by communicating with MSOs 402, OTT providers403, or content databases 401.

In another example, user 101 may indicate an authorization for socialnetworking system 160 to directly program the user's DVR to record TVshows or other content based on the posts of user 101. For example,social networking system 160 may determine from posts of user 101 thatthe user is interested in the show “Lone Survivor.” An example of a postby user 101 may be a comment such as “that show looks interesting” thatis made in response to a friend's post about the show “Lone Survivor.”If user 101 has authorized social networking system 160 to directlycontrol his DVR based on posts of user 101, social networking system 160sends instructions to the user's DVR to record the show “Lone Survivor”as described above.

In another example, user 101 may indicate an authorization for socialnetworking system 160 to directly control the user's STB to view TVshows or other content based on content that is shared with user 101 byother users (e.g., friends). For example, social networking system 160may determine that a friend of user 101 has shared the show “LoneSurvivor” with user 101. If user 101 has authorized social networkingsystem 160 to directly control his STB based on content shared by hisfriends, social networking system 160 may send instructions to the STBof user 101 to view the show “Lone Survivor.” This may include, forexample, tuning to the appropriate channel to view “Lone Survivor.”

In some embodiments, user 101 may indicate a preference that socialnetworking system 160 seeks the user's permission before directlyprogramming the user's DVR. For example, once social networking system160 determines that at least one friend 101 a-e of user 101 is currentlywatching or is planning to watch a particular TV show, social networkingsystem 160 may send an authorization message to user 101 recommendingthat user 101 records the particular TV show. As a specific example forillustrative purposes only, a particular friend 101 a-e of user 101 maypost the message “I can't wait to watch Lone Survivor tonight” to socialnetworking system 160. Social networking system 160 may then notify user101 that his friend is planning to watch “Lone Survivor.” In certainembodiments, the notification may be via email, text messaging, a poston social networking system 160, or any other appropriate method. Thenotification may also include a request for user 101 to authorize socialnetworking system 160 to automatically record “Lone Survivor” for theuser. For example, the notification may include a hyperlink, button, orany other appropriate method that allows the user to easily approvesocial networking system 160 to record the show. If the user takes theappropriate steps to authorize social networking system 160 to recordthe show, social networking system 160 sends instructions to the user'sDVR to record the show “Lone Survivor” as discussed above. If the userdoes not take the appropriate steps to authorize social networkingsystem 160 to record the show, social networking system 160 does notsend instructions to the user's DVR to record the show “Lone Survivor.”

In certain embodiments, user 101 may authorize one or more friends 101a-e to directly control what is recorded on the user's DVR. For example,user 101 may indicate to social networking system 160 that certainfriends 101 a-e are authorized to recommend shows to record on theuser's DVR. As a specific example for illustrative purposes only, user101 may indicate to social networking system 160 that friend 101 a isauthorized to record shows on the DVR of user 101. Friend 101 a may theninteract with social networking system 160 using, for example, a webbrowser or mobile application to indicate to social networking system160 that he would like for user 101 to view “Lone Survivor.” Forexample, friend 101 a may send a message to user 101 telling user 101the he is planning to watch “Lone Survivor” or inviting or inviting user101 to watch “Lone Survivor” that friend 101 a is currently watching. Insome circumstances, however, user 101 may not view the message orinteract with the invite before the show airs (or after a certain periodof time). In such circumstances, social networking system 160 maydetermine that user 101 will not be able to view or record the show andthus take action to record the show for user 101. To do this, socialnetworking system 160 may determine a particular start time, channel,and duration for the show “Lone Survivor” and then send instructions tothe DVR of user 101 to begin recording the determined channel at thedetermined start time and for the determined duration. As a result, user101 and friend 101 a may each view the same show and may subsequentlyinteract via social networking system 160 about the show. Thus, thesocial interaction between user 101 and friend 101 a may be improved.

In some embodiments, social networking system 160 may send instructionsto a user's STB to pause the current show that is being viewed by user101 based on interactions of the user with social networking system 160.For example, if user 101 is interacting with social networking system160 via another client system 130, social networking system 160 maydetect this interaction and send instructions to STB/gateway 404 c ofuser 101 to pause the current TV show that is being viewed. As oneexample for illustrative purposes only, user 101 may be interacting withsocial networking system 160 via a smartphone to, for example, postmessages to social networking system 160. If social networking system160 determines that user 101 has posted to social networking system 160more than a predetermined number of times within a predetermined periodof time, social networking system 160 may determine that user 101 is notcurrently viewing the particular TV show and thus send instructions toSTB/gateway 404 c of user 101 to pause the TV show. As another example,user 101 may be chatting with friends 101 a-e using a chat program ofsocial networking system 160. Social networking system 160 may detectthat user 101 is interacting with the chat program more than apredetermined amount of time or has typed more than a predeterminedamount of text into the chat program and thus send instructions toSTB/gateway 404 c of user 101 to pause any program that is currentlybeing viewed by the user.

In one embodiment, social networking system 160 may send instructions toSTB/gateway 404 c of user 101 to mute or pause the TV when user 101receives a phone call. For example, a mobile app of the socialnetworking system 160 running on a smartphone of user 101 may detectthat user 101 has received a phone call. The mobile app may then send anindication of the phone call to social networking system 160. Socialnetworking system 160 may then send instructions to STB/gateway 404 c ofuser 101 to pause or mute the current TV show. As a result, user 101 maybe able to focus on the phone call without having to worry aboutmanually pausing or muting the TV.

FIG. 7 illustrates an example method 700 for providing instructions to aSTB of a user based on the user's authorization. Method 700 may begin atstep 710 where authorization is received from a first user of asocial-networking system to control at least in part one or morefunctions of a STB of the first user. In some embodiments, the functionsmay include recording shows or providing setting to record shows,pausing or begin playing shows on a TV, muting or un-muting the volumeof the TV, turning on or off closed-captioning on the TV, turning on oroff power to the TV, changing the channel being viewed on the TV, andthe like. In certain embodiments, the received authorization from thefirst user may indicate an authorization for the social-networkingsystem to automatically record content on the user's STB based on whatthe user's friends are currently watching or are planning to watch. Incertain embodiments, the received authorization from the first user mayindicate an authorization for the social-networking system to recommendto the first user one or more shows to record based on what the user'sfriends are currently watching or are planning to watch. In certainembodiments, the received authorization from the first user may indicatean authorization for a friend of the first user to directly controlcontent recorded on the first user's STB. In some embodiments, the STBof the first user may refer to STB/gateway 404 c described above. Incertain embodiments, the STB may include a DVR. In some embodiments, thesocial-networking system refers to social networking system 160described above and includes a graph such as social graph 300 describedabove.

At step 720, one or more social actions of one or more second users ofthe social-networking system are determined. In certain embodiments, theone or more second users refer to friends 101 a-e described above. Insome embodiments, the one or more second users are associated with auser node in the social graph of step 710. In some embodiments, the usernodes of the one or more second users are connected to a user nodeassociated with the first user by one or more edges in the social graph.In certain embodiments, the determined social actions of step 720 arerelated to current or future content consumption by either the secondusers or the first user. For example, the determined social actions mayrefer to posts by the first or second users to the social-networkingsystem. In some embodiments, the posts may indicate particular contentsuch as a TV show or movie that the user is currently watching, isplanning to watch, is liked by the user, or looks interesting to theuser. In certain embodiments, the determined social actions of step 720refer to any interaction by the first user with the social-networkingsystem. For example, the social actions may refer to interacting with achat program of the social-networking system, posting messages to thesocial-networking system, receiving a phone call on a mobile device inwhich an app of the social-networking system is running, and the like.

At step 730, one or more instructions for the STB of the first user aredetermined based at least in part on the social actions of step 720. Forexample, in embodiments where the social actions of step 720 refer tocurrent or future content consumption such as posts that indicate that auser is currently watching or is planning to watch a particular TV show,the determined one or more instructions for the STB may includeinstructions to record or view the particular TV show. As anotherexample, in embodiments where the social actions of step 720 refer tointeractions with the social-networking system such as a user typing ina chat session, the determined one or more instructions for the STB mayinclude instructions to pause the TV show currently being viewed by theuser. As yet another example, in embodiments where the social actions ofstep 720 refer to a mobile app of the social-networking system detectingthat a user has received a phone call on a mobile device, the determinedone or more instructions for the STB may include instructions to pauseor mute the TV show currently being viewed by the user. While particularscenarios and instructions have been discussed, the disclosureanticipates any appropriate instruction being determined based on users'social actions.

At step 740, the instructions determined in step 730 are provided forthe STB of the first user. In one embodiment, the instructions areprovided by social networking system 160 directly to the DVR or STB ofthe first user via network 110. In some embodiments, the instructionsare provided by social networking system 160 to the DVR or STB of thefirst user via OTT providers 403 or MSOs 402. In certain embodiments,the instructions are provided to the DVR or STB of the first user viasocial TV dongle 810 described in more detail below. The disclosureanticipates any appropriate means of providing the instructions to theDVR or STB of the first user. After step 740, method 700 may end.

Particular embodiments may repeat one or more steps of the method ofFIG. 7, where appropriate. Although this disclosure describes andillustrates particular steps of the method of FIG. 7 as occurring in aparticular order, this disclosure contemplates any suitable steps of themethod of FIG. 7 occurring in any suitable order. Moreover, althoughthis disclosure describes and illustrates particular components,devices, or systems carrying out particular steps of the method of FIG.7, this disclosure contemplates any suitable combination of any suitablecomponents, devices, or systems carrying out any suitable steps of themethod of FIG. 7.

FIG. 8 illustrates a social TV dongle 810 that may be utilized in someembodiments of the network environment of FIG. 4. In general, social TVdongle 810 may be any combination of hardware and software that residesbetween a content source 820 and a TV 830. Social TV dongle 810 may becommunicatively (either bidirectional or unidirectional) coupled tocontent source 820 and TV 830 via, for example, an HDMI cable or anyother appropriate link for sending video streams 850 and 860. Social TVdongle 810 may also be communicatively coupled to one or more mobiledevices 840, social networking system 160, and OTT providers 403 such asNetFlix, Hulu, Amazon Video, YouTube, Spotify, and the like. Forexample, social TV dongle 810 may be wirelessly coupled to mobile device840 via BLUETOOTH, NFC, or any other appropriate wireless communicationsprotocol. Social dongle 810 may also be communicatively coupled tosocial networking system 160 and OTT providers 403 via for example,WiFi, Ethernet, or an HDMI cable. In some embodiments, social TV dongle810 may send instructions via IR to content source 820. In certainembodiments, social TV dongle 810 may be configured as a WiFi accesspoint. In some embodiments, social TV dongle 810 may be communicativelycoupled to a network such as the internet via an HDMI cable (e.g., anHDMI cable coupled to content source 820 or TV 830.)

In general, social TV dongle 810 receives a video stream 850 fromcontent source 820 and outputs a modified video stream 860 to TV 830.The modified video stream 860 may include social content from socialnetworking system 160. For example, social TV dongle 810 may overlayadvertisements from social networking system 160 onto video stream 850in order to create modified video stream 860 for display on TV 830. Asanother example, social TV dongle 810 may overlay (full screen orpartial screen) notifications, newsfeeds, tickers, or any otherinformation from social networking system 160 onto video stream 850 inorder to create modified video stream 860 for display on TV 830. Themodified video stream 860 may also include content from OTT providers403. For example, TV dongle 810 may overlay videos from NetFlix ontovideo stream 850 in order to create modified video stream 860 fordisplay on TV 830. A particular embodiment of social TV dongle 810 isdiscussed in more detail below in reference to FIG. 9. A particularembodiment of how social TV dongle 810 creates modified video stream 860for display on TV 830 is discussed in more detail below in reference toFIG. 11. In some embodiments, social TV dongle 810 may not be connectedto any content source 820 and may instead serve as the only input to TV830.

Content source 820 may be any device that outputs video stream 850 fordisplay on a TV 830. For example, content source 820 may be a STB or DVRof user 101 such as STB/gateway 404 c discussed above. In someembodiments content source 820 may be a gaming console such as an XBOX,Wii, or PLAYSTATION. In some embodiments, content source 820 may be anydevice from Roku, Boxee, Apple TV, or Google TV that allows users toaccess content from OTT providers 403 such as NetFlix, Hulu, AmazonVideo, YouTube, and the like.

TV 830 may be any device capable of displaying content such as videostreams from content source 820 to user 101. For example, TV 830 may beconnected device 404, TV 404 d, or unconnected device 405 discussedabove. Mobile device 840 may be any mobile computing device of user 101such as a smartphone, tablet computer, laptop computer, and the like. Insome embodiments, mobile device 840 is mobile device 404 b discussedabove.

In operation, social TV dongle 810 receives video stream 850 that isoutput from content source 820. Social TV dongle 810 creates modifiedvideo stream 860 from video stream 850 for display on TV 830. To createmodified video stream 860, some embodiments of social TV dongle 810overlay social content 870 from social networking system 160. Forexample, social TV dongle 810 may receive video stream 850 associatedwith a TV show via an HDMI cable from content source 820. Social TVdongle 810 may overlay social content 870 on top of video stream 850according to instructions received from social networking system 160before outputting modified video stream 860 to TV 830. As a result, user101 may be able to view social content 870 from social networking system160 on TV 830 while watching the TV show. Social content 870 may includean advertisement 870 a, notification 870 b, newsfeed 870 c, ticker 870d, chat session 870 e, or any other appropriate social content fromsocial networking system 160. Various embodiments of social content 870that may be displayed on TV 830 on top of a TV show are discussed inmore detail below. While various embodiments are discussed in referenceto viewing a TV show on TV 830, it should be understood that socialcontent 870 from social networking system 160 may be displayed by socialTV dongle 810 on TV 830 on top of any content (e.g., a video game, amovie, a blank screen, etc.).

In some embodiments, social TV dongle 810 may overly advertisements 870a from social networking system 160 on top of video stream 850 to createmodified video stream 860 for display on TV 830. In some embodiments,advertisements 870 a may take up the entire screen of TV 830. In otherembodiments, advertisements 870 a may take up only a portion of thescreen of TV 830. Advertisements 870 a may be any appropriateadvertisement. As one example, advertisement 870 a may a full-screenadvertisement from social networking system 160 that replaces abroadcast commercial in video stream 850 from, for example, MSO 402 orOTT provider 403. In such embodiments, social TV dongle 810 may analyzeincoming video stream 850 in order to determine when a broadcastcommercial in the video stream begins. Then, social TV dongle 810 mayoverlay a full-screen advertisement 870 a over the commercial. In oneembodiment, advertisement 870 a may be for a product that is currentlybeing displayed in a TV show. As an example for illustrative purposesonly, consider a TV show in which the product “Mega-Cola” is shown in aparticular scene. Social TV dongle 810 may receive an advertisement formsocial networking system 160 for Mega-Cola and display a smalladvertisement for Mega-Cola while the product is being shown in thescene. Particular embodiments of determining product placement in a TVshow are discussed further below in reference to FIG. 19.

In some embodiments, social networking system 160 may selladvertisements based on what users 101 have explicitly indicated theywill watch or record. For example, social networking system 160 mayanalyze posts to social networking system 160 to determine which usershave indicated they are planning to watch a particular show. In someembodiments, social networking system 160 may determine what shows users101 have instructed a DVR to record. Social networking system 160 maythen sell advertisements which are targeted to the group of users thatthe social networking system 160 determined are planning to watch theparticular show.

In some embodiments, social TV dongle 810 may overly notifications 870 bfrom social networking system 160 on top of video stream 850 to createmodified video stream 860 for display on TV 830. For example, if one ormore friends 101 a-e of user 101 have posted messages to socialnetworking system 160 about a particular show, the messages may bedisplayed on TV 830 while the user is watching the show. In someembodiments, notifications 870 b may be relevant to or related to thecurrent show being viewed on TV 830. For example, if a particular friendof user 101 has posted “this episode of Lone Survivor is really goodtonight,” a notification 870 b may appear on TV 830 of user 101 whilethe show “Lone Survivor” is being viewed. In some embodiments,notification 870 b may indicate one or more of the friend's name, thefriend's avatar, and the friend's message. For example, notification 870b may include “Friend A says: this episode of Lone Survivor is reallygood tonight.”

In certain embodiments, messages from friends 101 a-e are shown to user1010 at the point in a show in which the messages were posted,regardless of whether the show is being viewed in real-time ortime-delayed. To accomplish this, certain embodiments time-stamp poststo social networking system 160 with a time that is relative to aparticular show. For example, if a friend 101 a posts a comment about aparticular show fifteen minutes after the beginning of the show, thecomment may be time-stamped with a time that indicates it was postedfifteen minutes into the show. Then, if another user 101 is viewing arecorded version of the same show (i.e., a time-delayed version), thefriend's comment may appear at the same point in the show (i.e., fifteenminutes after the beginning of the show) for user 101. In embodimentswhere user 101 and friend 101 a are both viewing the same live show,social TV dongle 810 may display to user 101 posts by friend 101 a aboutthe show immediately after they are posted.

In one embodiment, social TV dongle 810 may overlay newsfeed 870 c,ticker 870 d, and chat session 870 e from social networking system 160on top of a TV show being viewed by user 101 on TV 830. Newsfeed 870 cmay refer to any list of social content from social networking system160 for user 101. For example, newsfeed 870 c may include friends'posts, friends' status updates, friends' photo, friends' likes, or anyother social content from social networking system 160 according tosocial graph 300. Ticker 870 d may refer to real-time updates of stocks,sports scores, or any other data of user 101. Chat sessions 870 e mayinclude messages exchanged between user 101 and one or more friends 101a-e using, for example, a chat interface on a website or mobile device840. Particular embodiments of chat sessions are discussed in moredetail below in reference to FIG. 23.

In some embodiments, social content 870 may be filtered to only displayitems that are relevant to the show the user is watching. For example,only content in newsfeed 870 c of user 101 that is relevant to theparticular show user 101 is currently watching may be displayed on topof the show. As another example, only content in ticker 870 d that isrelevant to the current sport that user 101 is viewing may be displayedon TV 830. For example, if user 101 is currently viewing a baseballgame, only baseball scores or baseball-related information from ticker870 d may be displayed on top of the current baseball game being viewedby user 101.

As discussed above, social networking system 160 may overlay broadcastcommercials with advertisements 870 a from social networking system 160.In some embodiments, social networking system 160 may provide a biddingscheme for displaying advertisements 870 a from social networking system160 for users on TV 830. For example, social networking system 160 mayallow MSO 402 to submit a bid to social networking system 160 to displaybroadcast advertisements on TV 830 instead of advertisements 870 a fromsocial networking system 160. Then, based on the bid of MSO 402, socialnetworking system 160 may determine whether or not to overlay abroadcast commercial with advertisements 870 a from social networkingsystem 160. As one example, if the bid from MSO 402 is not greater thana predetermined price, social networking system 160 may completelyoverlay broadcast commercials with advertisements 870 a from socialnetworking system 160.

In some embodiments, user 101 may “check in” with social networkingsystem 160 using social TV dongle 810. In some embodiments, user 101 may“check in” by interacting with social TV dongle 810 using mobile device840 to indicate the current location of user 101 to social networkingsystem 160. For example, user 101 may press a “check in” buttondisplayed in a mobile app of social networking system 160 on mobiledevice 840 when user 101 is in proximity to social TV dongle 810. SocialTV dongle 810 may then send information to social networking system 160that indicates that user 101 has checked in at home. As a result, a postsuch as “User 101 has checked in at home” may be posted to socialnetworking system 160 for user 101 and made available to friends 101 a-eof user 101. In another example, user 101 may visit the home of friend101 a and press a “Check in” button displayed in a mobile app of socialnetworking system 160 on mobile device 840 when user 101 is in proximityto the social TV dongle 810 of friend 101 a. The social TV dongle 810 offriend 101 a may then send information to social networking system 160that indicates that user 101 has checked in at the home of friend 101 a.As a result, a post such as “User 101 has checked in at the home offriend 101 a” may be posted to social networking system 160 for user 101and made available to friends 101 a-e of user 101.

FIG. 9 illustrates a particular embodiment of social TV dongle 810. Inthis embodiment, social TV dongle 810 includes a CPU 911, flash memory912, RAM 913, HDMI receiver 914, IR receiver 915, USB client 916, powermanagement IC (PMIC) 917, WiFi/BLUETOOTH transceiver 918, HDMItransmitter 919, IR transmitter 921 and USB host 922. HDMI receiver 914receives video stream 850 from content source 820 via an HDMI cable. IRreceiver 915 receives IR communications from a remote control device950. USB client 916 communicates with any device such as a PC via USB.PMIC 917 provides power to social TV dongle 810 via a USB cable pluggedinto a wall adapter or PC 960. WiFi/BLUETOOTH transceiver 918communicates with mobile device 840 and social networking system 160using the WiFi and BLUETOOTH, respectively. HDMI transmitter 919communicates modified video stream 860 to TV 830 via an HDMI cable. IRtransmitter 921 communicates IR instructions to STB/gateway 404 c. USBhost 922 provides USB communications to devices such as a web cam 970.Devices 914-922 are communicatively coupled to CPU 911 via, for example,a bus. CPU 911 controls the operation of social TV dongle 810 andperforms the functions of social TV dongle 810 described herein. While aparticular embodiment of social TV dongle 810 is illustrated in FIG. 9,social TV dongle 810 may include any appropriate components in anyappropriate configuration. In certain embodiments, social TV dongle 810may be fully or partially implemented in SW. In some embodiments, socialTV dongle 810 may be a low-cost implementation that includes a minimalamount of memory or processing power.

In some embodiments, social TV dongle 810 may be configured (e.g.,during an initial or installation process, etc.) by an appropriatemeans. In some embodiments, a quick response (QR) code may be providedby social networking system 160 in order to configure social TV dongle810. In some embodiments, user 101 may simply scan the QR code using,for example, mobile device 840, in order to configure social TV dongle810. In some embodiments, audio fingerprinting may be utilized in orderto configure social TV dongle 810. For example, social TV dongle 810 mayutilize voice recognition to identify user 101 and configure social TVdongle 810 accordingly.

FIG. 10 illustrates an example method 1000 for concurrently displayingsocial content with shows being viewed on a TV. Method 1000 may begin atstep 1010, where a broadcast video stream is received at a social TVdongle from a content source using a first interface. In someembodiments, the first interface may refer to HDMI receiver 914. In someembodiments, the broadcast video stream refers to video stream 850above. In certain embodiments, the broadcast video stream is received atsocial TV dongle 810. In some embodiments, the content source may referto content source 820 above. The broadcast video stream may includebroadcast TV shows, recorded shows or movies from a DVR, video gamedata, pay-per-view movies, content from OTT provider 403, or any othercontent to be viewed on a TV such as TV 830.

At step 1020, social content is received by the social TV dongle from asocial-networking system such as social networking system 160 using asecond interface. In certain embodiments, the second interface may referto WiFi/BLUETOOTH transceiver 918. In some embodiments, the socialcontent is received at a social TV dongle 810. In certain embodiments,the social content may refer to stories, message, posts, notifications,advertisements, newsfeeds, tickers, chat messages, or any otherinformation from the social-networking system. In some embodiments, thesocial content is related to a social graph such as social graph 300. Insome embodiments, the social content is filtered to include only socialcontent that is relevant to what is currently being viewed on a TV suchas TV 830.

At step 1030, a modified video stream is generated by the social TVdongle using the broadcast video stream received in step 1010 and atleast a portion of the social content received in step 1020. In someembodiments, the modified video stream is modified video stream 860discussed above. In certain embodiments, the modified video stream is acombination of the first video stream and the social content.

At step 1040, the modified video stream is sent for display on a displaydevice of a first user by the social TV dongle using a third interface.In certain embodiments, the third interface may refer to HDMItransmitter 919. In some embodiments, the modified video stream is sentby social TV dongle 810 to a TV such as TV 830. In other embodiments,the modified video stream is sent to any other appropriate displaydevice such as a computer monitor, a tablet computer, and the like.After step 1040, method 1000 may end.

Particular embodiments may repeat one or more steps of the method ofFIG. 10 where appropriate. Although this disclosure describes andillustrates particular steps of the method of FIG. 10 as occurring in aparticular order, this disclosure contemplates any suitable steps of themethod of FIG. 10 occurring in any suitable order. Moreover, althoughthis disclosure describes and illustrates particular components,devices, or systems carrying out particular steps of the method of FIG.10, this disclosure contemplates any suitable combination of anysuitable components, devices, or systems carrying out any suitable stepsof the method of FIG. 10.

FIG. 11 illustrates how particular embodiments of social TV dongle 810may create modified video stream 860 for display on TV 830. In thisembodiment, social TV dongle 810 receives video stream 850 from contentsource 820. Social TV dongle 810 may then decode incoming video stream850 into a series of incoming video frames 1120. Social TV dongle 810then overlays top frame 1130 onto incoming video frame 1120 to create acombined output frame 1110. Combined output frames 1110 are then sent asa modified video stream 860 for display on TV 830. Various embodimentsof top frame 1130 are discussed in more detail below.

In some embodiments, top frame 1130 is a composite overlay image that isplaced on top of incoming video frame 1120. In certain embodiments, topframe 1130 is a transparent window that includes a social area 1140.Social area 1140 may be a painted portion of top frame 1130 and maycomprise a portion of top frame 1130 (e.g., a small corner) or may fillup the entire top frame 1130. Social area 1140 may be opaque orsemi-transparent.

In some embodiments, social area 1140 may be painted directly onincoming video frame 1120. In such embodiments, social TV dongle 810 maynot create top frame 1130. Instead, any appropriate technology such asframe buffering may be utilized to draw social area 1140 directly onincoming video frame 1120. By drawing social area 1140 directly onincoming video frame 1120, social TV dongle 810 may create output frame1110 for display on TV 830.

In some embodiments, social area 1140 may include any social contentfrom social networking system 160 such as social content 870 discussedabove. For example, social area 1140 may include advertisements 870 a,notifications 870 b, newsfeeds 870 c, tickers 870 d, chat sessions 870e, or any other appropriate social content from social networking system160. In the illustrated embodiment of FIG. 11, for example, social area1140 includes social content from social networking system 160indicating that Friend A and two other friends of user 101 are currentlywatching the show being viewed by user 101. In some embodiments, socialarea 1140 may include a facepile (e.g., multiple photos of users ofsocial networking system 160).

In some embodiments, social TV dongle 810 may deduce what is importantin incoming video frame 1120 before determining where to place socialarea 1140. In general, social TV dongle 810 may perform this function inorder to avoid placing social area 1140 on top of an important area ofthe viewing screen. For example, if user 101 is viewing a singingcompetition show in which a voting telephone number is displayed on theTV screen, social TV dongle 810 may detect the telephone number andavoid placing social area 1140 on top of the telephone number or anylocation that would obstruct the telephone number. As another example,if user 101 is viewing a sporting event in which sports scores arescrolled along one edge of the TV screen, social TV dongle 810 maydetect the scrolling scores and avoid placing social area 1140 on top ofthe scores or any location that would obstruct the scores. Variousmethods of how social TV dongle 810 may determine important areas ofincoming video frame 1120 are discussed below.

In some embodiments, social TV dongle 810 may determine important areasof incoming video frame 1120 by determining static portions of thedisplay. For example, social TV dongle 810 may compare two or moresuccessive incoming video frames 1120 in order to determine portions ofthe incoming video frames 1120 that have not changed or have changedless than a predetermined amount. Social TV dongle 810 may determinethat the static portions are important areas and therefore may avoidplacing social area 1140 on top of the static areas. For illustrativepurposes only, consider a televised sporting event in which an area ofthe screen is dedicated to displaying the score. This dedicatedscoreboard typically would change very little, if any, betweensuccessive incoming video frames 1120. Social TV dongle 810 maydetermine the location of the scoreboard on the screen by comparingsuccessive incoming video frames 1120. Social TV dongle 810 may thenavoid overlapping the scoreboard with any portion of social area 1140.

In some embodiments, social TV dongle 810 may determine important areasof incoming video frame 1120 by determining any text shown on thedisplay. For example, social TV dongle 810 may analyze incoming videoframe 1120 in order to determine if any portion contains text. In someembodiments, this may be accomplished using optical characterrecognition (OCR) or any other appropriate technique. Social TV dongle810 may then avoid placing social area 1140 on top of the determinedtext. For illustrative purposes only, consider a televised singingcompetition in which a telephone number is displayed on the screen.Social TV dongle 810 may utilize, for example, OCR on incoming videoframes 1120 in order to determine that a telephone number is beingdisplayed and the location on the screen of the telephone number. SocialTV dongle 810 may then avoid overlapping the telephone number with anyportion of social area 1140.

In some embodiments, social TV dongle 810 may determine important areasof incoming video frame 1120 by determining whether any faces are shownon the display. For example, social TV dongle 810 may analyze incomingvideo frame 1120 using any appropriate facial recognition technology inorder to determine if any portion of the screen contains a face. SocialTV dongle 810 may then avoid placing social area 1140 on top of any ofthe determined faces. For illustrative purposes only, consider atelevised debate in which the faces of multiple people are displayed onthe screen. Social TV dongle 810 may utilize, for example, facialrecognition software on incoming video frames 1120 in order to determinethat multiple faces are being displayed on the screen and the locationon the screen of the faces. Social TV dongle 810 may then avoidoverlapping any of the faces with any portion of social area 1140.

In some embodiments, social TV dongle 810 may determine important areasof incoming video frame 1120 using metadata provided by contentproducers of the program being viewed. For example, social TV dongle 810may analyze metatdata embedded in incoming video frame 1120 thatindicates important areas of the screen. As another example, social TVdongle 810 may receive metadata indicating important areas of the screenfor shows from content producers such as MSOs 402. Social TV dongle 810may then avoid placing social area 1140 on top of any of the importantareas as indicated by the metadata.

In certain embodiments, the shape or location of social area 1140 may bedynamically adjusted based on the determined important areas of thescreen discussed above. For example, if social area 1140 is displayed ata first location at a first time in a show, it may be resized or movedat a later time in the show if social TV dongle 810 determines that thesocial area 1140 is overlapping an important area of the screen. Forillustrative purposes only, consider a televised singing competition inwhich a contestant performs during a first portion of the show and thena telephone number to vote for the contestant is displayed on the screenat a later point in the show. If social area 1140 is displayed in thelower center portion of the screen during the contestant's performanceduring the first portion of the show (e.g., to avoid overlapping thesinger's face), it may overlap the telephone number when it is displayedlater in the show. To avoid this, social TV dongle 810 may detect that anew important area of the screen has appeared (i.e., the telephonenumber) and either adjust the size or shape of social area 1140 to avoidthe telephone number, or move social area 1140 to avoid the telephonenumber.

Some embodiments of social TV dongle 810 may resize social area 1140 tomask or hide content in incoming video frames 1120. For example,incoming video frames 1120 may include an EPG from MSO 402. Social TVdongle 810 may detect the portion of the screen occupied by the EPGusing, for example, any of the techniques described above fordetermining important areas of the screen. Social TV dongle 810 may thenresize or move social area 1140 to mask the EPG.

In certain embodiments, incoming video frame 1120 may be shrunk orresized in output frame 1110. For example, incoming video frame 1120 maybe shrunk to only occupy a portion of output frame 1110 while socialarea 1140 occupies the remaining output frame 1110. This may beutilized, for example, by social TV dongle 810 to display all or aportion of a programming guide that includes social content from socialnetworking system 160.

Some embodiments of social TV dongle 810 may mask blank or corruptedincoming video frames 1120 due to slow channel switching of contentsource 820. For example, some STBs are notoriously slow at switchingchannels. This typically results in the extended display of blank orcorrupted frames of video. Some embodiments of social TV dongle 810 maymask this effect and appear to speed up the latency of changingchannels. To accomplish this, some embodiments of social TV dongle 810may fade top frame 1130 in and out of being opaque when social TV dongle810 detects a channel change in incoming video stream 850. For example,when social TV dongle 810 detects a channel change in incoming videostream 850, it may fade the entire top frame 1130 to black. Then, aftera predetermined period of time or when social TV dongle 810 detects thatthe channel change in complete, top frame 1130 may be faded from blackback to transparent. In some embodiments, social TV dongle 810 maydisplay social area 1140 on top frame 1130 while it is opaque. As aresult, a user may perceive that the change in channels occurred fasterthan it actually occurred.

In some embodiments, social area 1140 may include reminders from acalendar associated with user 101. For example, user 101 may storecalendar events with social networking system 160. Social networkingsystem 160 may cause reminders for calendared events to appear in socialarea 1140 at any appropriate time. In some embodiments, socialnetworking system 160 may cause TV 830 to turn on in order to display acalendar reminder in social area 1140.

FIG. 12 illustrates an example method 1200 for displaying social contenton top of shows currently being viewed on a TV. Method 1200 may begin atstep 1210, where a first video stream is received from a content source.In some embodiments, the video stream refers to video stream 850 above.In certain embodiments, the video stream is received at social TV dongle810. In some embodiments, the content source may refer to content source820 above. The video stream may include broadcast TV shows, recordedshows or movies from a DVR, video game data, pay-per-view movies,content from OTT provider 403, or any other content to be viewed on a TVsuch as TV 830.

At step 1220, content from a social-networking system such as socialnetworking system 160 is accessed. In some embodiments, the content isreceived at a social TV dongle 810. In certain embodiments, the contentmay refer to social content such as stories, messages, posts,notifications, advertisements, newsfeeds, tickers, chat messages, or anyother information from the social-networking system. In someembodiments, the content is related to a social graph such as socialgraph 300. In some embodiments, the social content is filtered toinclude only content that is relevant to what is currently being viewedon a TV such as TV 830. In some embodiments, the content may be content(e.g., video content) from OTT provider 403.

At step 1240, an overlay area is generated. In some embodiments, theoverlay area includes the content accessed in step 1220 such as stories,messages, posts, notifications, advertisements, newsfeeds, tickers, chatmessages, or any other information from the social-networking system. Insome embodiments, the generated overlay area may refer to social area1140 described above.

At step 1250, a placement for the overlay area of step 1240 isdetermined. In some embodiments, the placement may be based on adetermination of important areas of the first video stream of step 1210,as described above. For example, the overlay area may be placed to ovoidany determined important areas of the first video stream such as facesor text. In some embodiments, the overlay area may be dynamicallyresized to avoid any determined important areas, as described above.

At step 1250, a modified video stream is generated using the first videostream received in step 1010 and the overlay area of step 1230. In someembodiments, the modified video stream is combined output frame 1110. Incertain embodiments, the modified video stream is generated by combiningincoming video frames 1120 with top frames 1130, as described above. Insuch embodiments, the top frame 1130 includes the overlay area of step1240. In some embodiments, the modified video stream is generated bypainting the overlay area of step 1240 directly on incoming video frames1120 of the first video stream.

At step 1260, the modified video stream is sent for display on a displaydevice of a first user. In some embodiments, the modified video streamis sent by social TV dongle 810 to a TV such as TV 830. In otherembodiments, the modified video stream is sent to any other appropriatedisplay device such as a computer monitor, a tablet computer, and thelike. After step 1240, method 1200 may end.

Particular embodiments may repeat one or more steps of the method ofFIG. 12 where appropriate. Although this disclosure describes andillustrates particular steps of the method of FIG. 12 as occurring in aparticular order, this disclosure contemplates any suitable steps of themethod of FIG. 12 occurring in any suitable order. Moreover, althoughthis disclosure describes and illustrates particular components,devices, or systems carrying out particular steps of the method of FIG.12, this disclosure contemplates any suitable combination of anysuitable components, devices, or systems carrying out any suitable stepsof the method of FIG. 12.

FIG. 13 illustrates an environment in which social TV dongle 810 orsocial networking system 160 determines what user 101 is watching on TV830. As discussed further below, social TV dongle 810 or socialnetworking system 160, either alone or in combination, may utilizevarious methods of determining what user 101 is watching on TV 830.These methods may include acoustic fingerprinting, analyzing electricalinterference, analyzing signals on an HDMI cable, analyzingclosed-captioning, analyzing image stills from an incoming video stream,and analyzing explicit signals from user 101. In embodiments wheresocial TV dongle 810 determines what user 101 is currently watching,social TV dongle 810 may communicate to social networking system 160what user 101 is viewing and a time associated with the viewing.

In some embodiments, acoustic fingerprinting is used to determine whatuser 101 is watching on TV 830. For example, while user 101 is watchingTV 830, a mobile app associated with social networking system 160 may berunning on mobile device 840 of user 101. Mobile device 840 may captureenvironmental sounds 1330 from the environment using a microphone 842.Environmental sounds 1330 may include, for example, audio from the showbeing viewed on TV 830. Environmental sounds 1330 are then sent tosocial networking system 160, either directly or via social TV dongle810. Social networking system 160 may then analyze environmental sounds1330 in order to determine what user 101 is watching on TV 830. Forexample, social networking system 160 may compare environmental sounds1330 to audio of known TV shows. Based on the comparison, socialnetworking system 160 may determine what show user 101 is watching on TV830.

In some embodiments, social networking system 160 may utilizeenvironmental sounds 1330 received from multiple users 101 in order todetermine whether the multiple users 101 are watching a TV showseparately or together. For example, if two users 101 are in the sameroom while watching the same TV show, environmental sounds 1330 capturedby each users' mobile device 840 may be similar. Social networkingsystem 160 may compare environmental sounds 1330 captured from each userand determine that the two users are located in the same environment ifthe environmental sounds 1330 are substantially similar. Additionally oralternatively, social networking system 160 may compare GPS or otherlocation data sent by the users' mobile devices 840 in order todetermine that two or more users are watching the same TV show at thesame location. In some embodiments, social networking system 160 maypost content to social networking system 160 indicating that the twousers are watching the same TV show at the same location.

In some embodiments, social networking system 160 may utilize phasedelay to determine what user 101 is watching. For example, most cableproviders utilize a unique time delay in sending signals to customers.Social networking system 160 may determine this delay by, for example,analyzing environmental sounds 1330 or signals from an HDMI cable. Oncethe delay has been determined, social networking system 160 may compareit to known delays of content providers. For example, if the delay isdetermined to be 102 ms, social networking system 160 may determine thatuser 101 is watching content from Comcast if Comcast has a known delayof 102 ms. Once a content provider is determined, social networkingsystem 160 may determine what user 101 is watching by cross-referencingenvironmental sounds 1330 with known schedules of content for thecontent provider (e.g., if user 101 is determined to be watching Comcastat 6 PM, environmental sounds 1330 may be compared to audio of showsbeing offered by Comcast at 6 PM).

In some embodiments, electrical interference is utilized to determinewhat user 101 is currently watching. For example, social TV dongle 810may be plugged into an electrical outlet 1310 in the home of user 101.TV 830 may also be plugged into the same electrical outlet 1310, or anyanother electrical outlet 1310 in the home of user 101. Social TV dongle810, or any other appropriate device plugged into an electrical outlet1310 in the user's home, may capture electrical interference (e.g.,noise) present on a power cord plugged into electrical outlet 1310. Thiselectrical interference may be introduced into the electrical wiring ofthe user's home by, for example, TV 830. The electrical interference maybe due to variance in the electrical load from TV 830 caused by varyingsounds or volumes of a TV show being displayed. In some embodiments, theelectrical interference may be unique for each TV show. This electricalinterference pattern may be captured and sent to social networkingsystem 160. Social networking system 160 may analyze the capturedelectrical interference pattern and compare it to a database ofelectrical interference patterns or fingerprints for known TV shows. Bycomparing the captured electrical interference pattern with the storedelectrical interference fingerprints, social networking system 160 maybe able match the captured pattern with the pattern of a TV show andthus determine what user 101 is watching on TV 830.

In some embodiments, closed-captioning may be utilized to determine whatuser 101 is watching on TV 830. For example social TV dongle 810 mayreceive video stream 850 from content source 820 that containsclosed-captioning information 1320. Social TV dongle 810 may analyzevideo stream 850 and capture closed-captioning information 1320. Thecaptured closed-captioning information 1320 may then be sent to socialnetworking system 160 where it may be analyzed and compared toclosed-captioning data of known TV shows. Based on the comparison,social networking system 160 may match the captured closed-captioninginformation 1320 with closed-captioning of a known TV show and thusdetermine what user 101 is watching on TV 830.

In some embodiments, explicit signals from a content provider may beutilized to determine what user 101 is watching on TV 830. For examplesocial TV dongle 810 may receive video stream 850 from content source820 that contains data provided by the content provider via an API. Thedata may include information that identifies the content being displayed(e.g., title, episode number, time, date, etc.). Social TV dongle 810may analyze video stream 850 and capture the API data from the contentprovider. The data may then be sent to social networking system 160where it may be used to determine what user 101 is watching on TV 830.

In some embodiments, social networking system 160 may determine what oneor more users 101 are or will be watching by analyzing explicit signalsfrom the users. For example, social networking system 160 may analyzeposts of user 101 to social networking system 160 in order to determinewhat user 101 is or will be watching. As one example, if user 101 posts“I can't wait to watch ‘Lone Survivor’ tonight,” social networkingsystem 160 may determine that user 101 is planning to watch the show“Lone Survivor.” As another example, if two users are chatting about theshow “Lone Survivor,” social networking system 160 may determine thatthe users are planning to watch the show “Lone Survivor.”

In some embodiments, signals on an HDMI cable are utilized to determinewhat user 101 is currently watching. For example, social TV dongle 810may capture electrical patterns of signals travelling across an HDMIcable from content source 820. These electrical patterns may be uniquefor each TV show. The electrical patterns may be captured and sent tosocial networking system 160. Social networking system 160 may analyzethe captured electrical patterns and compare them to a database ofelectrical patterns for known TV shows. By comparing the capturedelectrical patterns with the known electrical patterns, socialnetworking system 160 may be able match the captured patterns with thepatterns of a TV show and thus determine what user 101 is watching on TV830.

In some embodiments, image stills from an incoming video stream areanalyzed to determine what user 101 is currently watching. For example,social TV dongle 810 may capture one or more incoming video frames 1120from video stream 850 received from content source 820. In someembodiments, the captured incoming video frames 1120 may be sent tosocial networking system 160. Social TV dongle 810 or social networkingsystem 160 may analyze the captured incoming video frames 1120 andcompare them to a database of image stills for known TV shows. Bycomparing the captured incoming video frames 1120 with the known imagestills, social networking system 160 or social TV dongle 810 may be ablematch the captured incoming video frames 1120 with the image stills of aTV show and thus determine what user 101 is watching on TV 830.

In some embodiments, social networking system 160 may, in response todetermining what user 101 is currently viewing on TV 830 using thetechniques discussed above, post content to social network 300indicating what user 101 is currently watching. For example, socialnetworking system 160 may post “User 101 is currently watching ‘LoneSurvivor’” based on a determination using environmental sounds 1330 orany other technique described above. In certain embodiments, socialnetworking system 160 may determine whether to automatically post whatuser 101 is currently watching based on preferences of user 101.

In some embodiments, social network system 160 may additionally utilizeGPS or other location data received from mobile device 840 to determinewhere user 101 is watching a show. In some embodiments, this data mayalso be used when posting content to social network 300. For example,social networking system 160 may compare GPS coordinates received frommobile device 840 to known GPS coordinates of the home of user 101.Then, when social networking system 160 posts content to social network300 regarding what user 101 is currently watching, it may includeinformation about where user 101 is watching the show. For example, ifthe received GPS coordinates correspond to a home address of user 101,social networking system 160 may post “User 101 is watching ‘LoneSurvivor’ at his home.”

In some embodiments, social TV dongle 810 may utilize a microphone orwebcam 970 in order to determine interesting points within a show beingviewed on TV 830. For example, social TV dongle 810 may perform audiosniffing using a coupled microphone in order to determine increasedsounds such as laughter or shouting within a room. Increase sounds fromusers 101 may be an indication of interesting points of a program suchas a funny moment, a scary moment, a shocking moment, and the like. Asanother example, social TV dongle 810 may utilize webcam 970 todetermine movement of viewers within the room. Large movements of users101 such as jumping may be an indication of interesting points of aprogram such as a touchdown in a football game. Once social TV dongle810 determines that an interesting point has occurred, it may send anindication to social networking system social networking system 160.Social networking system 160 may then correlate the indication with ascene or event within the viewed program.

Once it is determined what user 101 is watching on TV 830, anyappropriate action may be taken by social networking system 160. In someembodiments, social networking system 160 may add appropriate edges 306to social graph 300. For example, if social networking system 160determines that user 101 is watching the show “Lone Survivor,” an edge306 of “watched” may be added between user node 302 for user 101 andconcept node 304 for “Lone Survivor.” In some embodiments, socialnetworking system 160 may perform advertisement augmentation afterdetermining that user 101 is viewing an advertisement on TV 830. Forexample, if social networking system 160 determines that user 101 isviewing an advertisement for a specific product on TV 830, anotheradvertisement for that same product may appear in the newsfeed for user101 (e.g., in a web browser on the user's computer or in a mobile app onthe user's mobile device 404 b.). In some embodiments, a catalog for aspecific brand or a website of a product may be displayed on mobiledevice 404 b in response to user 101 viewing a commercial for the brandor product. In some embodiments, an advertisement for a specific productmay be displayed in a web browser (e.g., on mobile device 404 b or onany computing device) in response to determining that user 101 iswatching a particular program. For example, if it is determined thatuser 101 is watching a show that is typically watched by a certaindemographic, an advertisement for a product directed to that demographicmay be displayed to user 101.

FIG. 14 illustrates an example method 1400 for determining what a useris currently watching. Method 1400 may begin at step 1410, where dataassociated with content being viewed by a user of a social-networkingsystem is captured. In some embodiments, the content may refer to TVprogramming such as a TV show or movie. In some embodiments, the contentmay refer to a video game. The data may be captured by any appropriatedevice. In some embodiments, the data is captured by social TV dongle810 described above. In certain embodiments, the data is captured by amobile device such as mobile device 404 b. The data may refer to anyappropriate data associated with content being viewed by the user. Insome embodiments, the data may refer to: acoustic data such as dataassociated with environmental sounds 1130; data associated withelectrical interference present on an electrical circuit; dataassociated with signals on an HDMI cable; closed-captioning information;image stills from an incoming video stream; or explicit data posted bythe user to the social-networking system.

At step 1420, the data of step 1410 is accessed and analyzed. In someembodiments, the data of step 1410 is analyzed by social TV dongle 810.In certain embodiments, the data of step 1410 is analyzed by socialnetworking system 160.

At step 1430, the content being viewed by the user is identified fromthe captured data. In some embodiments, the content is identified bycomparing captured acoustic data such as environmental sounds 1130 withsounds of known TV shows. In some embodiments, the content is identifiedby comparing captured electrical interference patterns on an electricalcircuit with known electrical interference patterns of TV shows. In someembodiments, the content is identified by comparing captured signals onan HDMI cable with HDMI signals of known shows. In some embodiments, thecontent is identified by comparing captured closed-captioninginformation with closed-captioning information of known shows. In someembodiments, the content is identified by comparing captured imagestills from an incoming video stream to image stills of known shows. Insome embodiments, the content is identified by extracting show titles orother information from explicit data posted by the user to thesocial-networking system. At step 1440, some embodiments send anindication of the identified content being viewed by the user to thesocial-networking system. In certain embodiments, other information suchas the identity of the user and a time associated with the viewing maybe sent to the social-networking system.

Particular embodiments may repeat one or more steps of the method ofFIG. 14, where appropriate. Although this disclosure describes andillustrates particular steps of the method of FIG. 14 as occurring in aparticular order, this disclosure contemplates any suitable steps of themethod of FIG. 14 occurring in any suitable order. Moreover, althoughthis disclosure describes and illustrates particular components,devices, or systems carrying out particular steps of the method of FIG.14, this disclosure contemplates any suitable combination of anysuitable components, devices, or systems carrying out any suitable stepsof the method of FIG. 14.

FIG. 15 illustrates an embodiment that determines identities of one ormore users 101 who are watching content on TV 830. In some embodiments,social TV dongle 810 determines the identities of the users 101 who arein close proximity to TV 830 by capturing information from the users'mobile devices 840 a-c. The captured information may be sent to socialnetworking system 160 where it may be analyzed and compared to storeduser information. Then, social networking system 160 may take actionsbased on the identified users 101. For example, social networking system160 may cause TV 830 to automatically turn on and display content forthe identified users as soon as the identified users enter the room orcomes within proximity to TV 830. The displayed content may be, forexample, a TV show that one or more friends 101 a-e of the identifiedusers are currently watching. In some embodiments, an acknowledgement ofthe identified user may be displayed on TV 830 (e.g., if “User A” isidentified as coming within close proximity to TV 830, social TV dongle810 may turn on TV 830 and display “User A” or a profile picture of UserA on TV 830).

In some embodiments, social TV dongle 810 may determine who is in closeproximity to TV 830 by communicating with mobile devices 840 via, forexample, WiFi or BLUETOOTH. For example, social TV dongle 810 maycommunicate with one or more mobile devices 840 in order to capture aunique identifier 1510 associated with the mobile device 840. In oneexample, the unique identifier 1510 may be a Media Access Control (MAC)address or any other similar unique identifier associated with mobiledevice 840. Social TV dongle 810 may then communicate the capturedunique identifier 1510 of the one or more mobile devices 840 to socialnetworking system 160.

Once social networking system 160 receives unique identifier 1510 fromsocial TV dongle 810, some embodiments may use unique identifier 1510 toidentify the users watching TV 830. In one example, users 101 may havethe unique identifiers 1510 of their devices stored with socialnetworking system 160. Social networking system 160 may then compare thestored unique identifiers with the received unique identifier 1510 fromsocial TV dongle social TV dongle 810 in order to identify the userswatching TV 830.

Social networking system 160 may take any action based on the determinedidentities of the users within proximity to TV 830. In one example,social networking system 160 may send instructions to social TV dongle810 to display social media content on TV 830 based on the identity ofthe user. In another example, social TV dongle 810 may automaticallyturn on TV 830 when it detects that a particular user 101 has enteredthe room or has come within close proximity to TV 830 or social TVdongle 810. Social TV dongle 810 may then display programming such asweather, traffic, or a show on TV 830 that one or more friends of theidentified user are currently watching.

In some embodiments, social networking system 160 may perform automaticparental control of content viewable on TV 830 based on the identitiesof the users within proximity to TV 830. For example, social networkingsystem 160 may determine that the identities of all users viewing TV 830are less than a predetermined age. This may be determined using, forexample, user data stored with social networking system 160. Socialnetworking system 160 may then take action to block any adult orotherwise objectionable content on TV 830. In some embodiments, adultsassociated with a particular TV 830 may indicate to social networkingsystem 160 certain channels or content that should be blocked when noadult is within proximity to TV 830.

In some embodiments, a motion sensing input device 1520 such as a KINECTor a camera such as webcam 970 may be utilized to determine when usersare in close proximity to TV 830. For example, motion sensing inputdevice 1520 may be coupled to social TV dongle 810 via USB or any otherappropriate method and may send an indication 1522 to social TV dongle810 when one or more users have come within close proximity to TV 830 orhave left the room. Social TV dongle 810 may then take any appropriateaction such as turning TV 830 on or off. In some embodiments, motionsensing input device 1520 may also be utilized to determine useridentities. For example, when motion sensing input device 1520 detectsmovement, webcam 970 may capture images of users within close proximityto TV 830. These images may be sent to social networking system 160where they may be compared to images associated with users of socialnetworking system 160. Based on the comparison, social networking system160 may identify the users in close proximity to TV 830 and take anyappropriate action described above.

In certain embodiments, a camera such as webcam 970 may be utilized bysocial TV dongle 810 to determine when users are in the room and whetherthe users are actually watching TV 830. For example, social TV dongle810 or social networking system 160 may analyze images captured bywebcam 970 in order to determine how many users are in the room. Asanother example, social TV dongle 810 or social networking system 160may analyze images captured by webcam 970 in order to determine how manyusers leave the room when, for example, a commercial begins showing onTV 830. In certain embodiments, social TV dongle 810 or socialnetworking system 160 may analyze images captured by webcam 970 in orderto determine which scenes or advertisements get the most attention byusers 101. For example, images from webcam 970 may be analyzed todetermine if users are using mobile device 840 at any point during ashow or advertisement. In certain embodiments, social TV dongle 810 orsocial networking system 160 may analyze images captured by webcam 970in order to determine the identity of users 101, subject to privacysettings. For example, facial recognition or body-size recognition maybe utilized to compare captured images of users 101 with imagesassociated with users 101 on social networking system 160 (e.g., aprofile picture).

In some embodiments, a camera of a second screen of user 101 may beutilized to determine the identity of user 101. For example, mobiledevice 840 may include a front-facing camera that is utilized to capturean image of user 101 that may be used to identify user 101, subject toprivacy settings. In certain embodiments, mobile application 602captures an image of user 101 and sends the image to social TV dongle810 or social networking system 160. The image may be analyzed using,for example, a facial recognition process, in order to determine anidentity of user 101. Any appropriate action may then be performedaccording to the identity of user 101.

As one example of using a camera of a second screen to identity user101, user 101 may sit down to watch TV 830 while holding a mobile device840. Once user 101 selects a program from, for example, a socialprogramming guide discussed below in reference to FIG. 31, a photo ofuser 101 is captured using the mobile device 840 of user 101. Thecaptured image may then be analyzed using facial recognition in order todetermine the identity of user 101. Once the identity of user 101 isdetermined, actions such as publishing content to social graph 300 maybe performed. For example, if it is determined that the name or identityof user 101 is “User A” after user 101 selects the program “LoneSurvivor” to watch from the social programming guide, a post of “User Ais watching Lone Survivor” may be published to social graph 300, subjectto privacy settings.

As another example, user 101 may launch a mobile app on mobile device840 in order to view a social programming guide as discussed below inreference to FIG. 31. Once user 101 launches the mobile app, a photo ofuser 101 may be captured using the mobile device 840 of user 101. Thecaptured image may then be analyzed using facial recognition in order todetermine the identity of user 101. Once the identity of user 101 isdetermined, a customized social programming guide for the identifieduser may be presented. For example, a social programming guide that iscustomized with content for “User A” may be presented when it isdetermined that the identity of user 101 is “User A.”

In some embodiments, social TV dongle 810 may utilize gaze detection todetermine a user's interest. For example, webcam 970 or any othercoupled camera or gaze-detection device may be utilized by socialnetworking system 160 to track movements of the eyes of user 101. SocialTV dongle 810 may then determine what areas of the screen the user iswatching. This may be used to determine, for example, whether the useris looking at a product placed in a show. As another example, the gazedetection may be used to determine whether the user is looking at thescreen at all. If the gaze detection determines that a user is notlooking at TV 830 during a commercial, social networking system 160 mayinfer that the user is not interested in the product in theadvertisement. This data may then be provided to the advertiser for afee. In some embodiments, if gaze detection determines that a user isnot watching the current program, social TV dongle 810 may present anindication to user 101 to, for example, change the current channel orturn off TV 830. In some embodiments, if gaze detection determines thata user is intensely watching the current program, social TV dongle 810may, for example, take action to increase how long information ispresented to user 101 on TV 830 (e.g., slow down a ticker that isscrolling across the screen of the TV).

In certain embodiments, social TV dongle 810 may determine whether auser has performed a gesture and take any appropriate action based onthe determined gesture. For example, social TV dongle 810 may utilizeany device such as a KINECT or a camera such as webcam 970 to determinethat a user has performed a “shhhh” gesture (i.e., holding an indexfinger up to the user's lips). In response to determining the user hasperformed this gesture, social TV dongle 810 may take any appropriateaction such as lowering the volume of TV 830, muting the volume of TV830, or turning off TV 830. Examples of other possible gestures mayinclude gestures to change the channel of TV 830, turn up the volume ofTV 830, turn on TV 830, and the like.

In some embodiments, social networking system 160 may utilize otheranalytics to infer interests of user. For example, social networkingsystem 160 may detect which users are watching a particular show basedon information from social graph 300 (e.g., a user explicitly sharesthat he is watching the particular show). Social networking system 160may then analyze location information such as GPS data from mobiledevices 840 in order to determine the location of friends 101 a-e of theuser. Social networking system 160 may determine that all of the user'sfriends within a predetermined radius of the user can be inferred aswatching the same show as the user.

In some embodiments, social TV dongle 810 may display content fromsocial networking system 160 when it is determined that no user is inthe room or when it is determined that users are not watching TV 830.For example, when social TV dongle 810 determine that no users arewatching TV 830, social TV dongle 810 may display content such as adashboard, a stock ticker, photos from friends, notifications, or anyother social content from social networking system 160.

FIG. 16 illustrates an example method 1600 for determining identities ofone or more users who are watching content on a TV. The method may beginat step 1610, where a unique identifier associated with a mobile deviceof a user is received. In some embodiments, the unique identifier isreceived by social networking system 160 from social TV dongle 810. Insome embodiments, the unique identifier may be a MAC address or anyother similar unique identifier associated with a mobile device. In someembodiments, the user is in proximity to the social TV dongle 810. Insome embodiments, the user is viewing content on a display device suchas TV 830.

At step 1620, user data comprising information about users of asocial-networking system user is accessed. In some embodiments, the userdata is a data repository of social networking system 160. In someembodiments, the user data may be social graph 300. The user data maystore unique identifiers associated with devices of the users of thesocial-networking system.

At step 1630, the user data is compared with the received uniqueidentifier. In some embodiments, this may refer to searching a databaseof the social-networking system to determine whether the received uniqueidentifier matches stored unique identifier of the users.

At step 1640, an identity of the user is determined based on thecomparison in step 1630. For example, if the received unique identifiermatches a stored unique identifier for a particular user, it may bedetermined that the particular user owns the mobile device of step 1610.Thus, it may be determined that the particular user is currentlywatching the display device.

Particular embodiments may repeat one or more steps of the method ofFIG. 16, where appropriate. Although this disclosure describes andillustrates particular steps of the method of FIG. 16 as occurring in aparticular order, this disclosure contemplates any suitable steps of themethod of FIG. 16 occurring in any suitable order. Moreover, althoughthis disclosure describes and illustrates particular components,devices, or systems carrying out particular steps of the method of FIG.16, this disclosure contemplates any suitable combination of anysuitable components, devices, or systems carrying out any suitable stepsof the method of FIG. 16.

FIG. 17 illustrates an embodiment of social TV dongle 810 that maydetermine one or more user products 1710 of user 101. In the illustratedembodiment, user devices 1710 may include a personal computer 1710 a, amobile device 1710 a such as a smartphone, a laptop computer 1710 c, agaming console 1710 d, or any other appropriate user device. Userdevices 1710 communicate with network 110 or social TV dongle 810 via awireless communications protocol such as WiFi or BLUETOOTH.

In general, social TV dongle 810 may identify user products 1710 of user101 by wirelessly communicating either directly with user devices 1710or via network 110. For example, social TV dongle 810 may utilize WiFior BLUETOOTH sniffing to capture uniquely identifying information fromeach user device 1710. The information captured from user devices 1710may include, for example, a MAC address, a serial number, a uniqueproduct identification number, or any other data that may be used toidentify the specific models or manufacturers of user devices 1710. Oncecaptured, the unique identifiers of user devices 1710 may be analyzed bysocial TV dongle 810 or sent to social networking system socialnetworking system 160 for analysis. In some embodiments, the capturedunique identifiers may be compared to a database of unique identifiersin order to identify user devices 1710.

Social networking system 160 may utilize the information obtained fromuser devices 1710 to deliver specific content to TV 830. For example, ifsocial networking system 160 determines that user 101 is utilizing anIPHONE as mobile device 1710 b, social networking system 160 maycommunicate to social TV dongle 810 to display advertisements on TV 830directed to IPHONE users. As another example, if social networkingsystem 160 determines that user 101 is utilizing an XBOX as gamingconsole 1710 d, social networking system 160 may communicate to socialTV dongle 810 to display advertisements on TV 830 directed to XBOXusers. For example, social networking system 160 may causeadvertisements for specific XBOX games to be displayed on TV 830.

In some embodiments, signals on an HDMI cable may be utilized by socialTV dongle 810 to identify equipment such as STB/gateway 404 c or agaming consoler coupled either directly or indirectly to social TVdongle social TV dongle 810. For example, some devices send a uniqueidentification in signals sent via HDMI. Social TV dongle 810 maycapture this information from the HDMI cable and utilize it to determinewhat equipment is being used by user 101 as described above.

FIG. 18 illustrates an example method 1800 for determining one or moreuser products. The method may begin at step 1810, where a uniqueidentifier associated with a user product of a user is received. In someembodiments, the unique identifier is received by social networkingsystem 160 from social TV dongle 810. In some embodiments, the uniqueidentifier may be a MAC address or any other similar unique identifierassociated with a user device. In some embodiments, the user is inproximity to the social TV dongle 810. In some embodiments, the user isviewing content on a display device such as TV 830. The user product mayrefer to any other product that user owns. For example, the user productmay be a laptop computer, a smartphone, a gaming console, and the like.

At step 1820, a database of known user products is accessed. Thedatabase may include unique identifiers of known products. At step 1830,the received unique identifier is compared to the database of knownproducts. Based on this comparison, an identity of the user product maybe determined in step 1840. For example, it may be determined that theuser owns a specific gaming console.

At step 1850, content related to the identified user product may bepresented to the user. For example, if it is determined in step 1840that the user owns a specific gaming console, advertisements for gamesfor that gaming console may be presented to the user. As anotherexample, if it is determined in step 1840 that the user owns a specificbrand of laptop, advertisements for other products with that brand maybe presented to the user.

Particular embodiments may repeat one or more steps of the method ofFIG. 18, where appropriate. Although this disclosure describes andillustrates particular steps of the method of FIG. 18 as occurring in aparticular order, this disclosure contemplates any suitable steps of themethod of FIG. 18 occurring in any suitable order. Moreover, althoughthis disclosure describes and illustrates particular components,devices, or systems carrying out particular steps of the method of FIG.18, this disclosure contemplates any suitable combination of anysuitable components, devices, or systems carrying out any suitable stepsof the method of FIG. 18.

FIG. 19 illustrates embodiments that may determine products placed inprograms being viewed on TV 830. In general, many TV shows and moviesutilize product placement to show specific products to users viewingthat show. For example, a particular episode of a TV show may show anactor consuming a specific product such as “Mega-Cola” as illustrated onthe screen of TV 830 in FIG. 19. As another example, a scene of a moviemay show actors utilizing a laptop in which the logo of the manufacturerof the laptop is visible. In certain embodiments, social TV dongle 810or social networking system 160 may utilize various methods, such asthose described below, to determine which products are being shown on TV830. In some embodiments, social TV dongle 810 or social networkingsystem 160 may take any appropriate action once a determination is madethat a specific product is being shown on TV 830. For example, a websiteor mobile app of the determined product may appear on mobile devices 840of users determined to be viewing TV 830 as described above.

In some embodiments, closed-captioning information may be utilized todetermine specific products being shown on TV 830. For example, socialTV dongle 810 may capture closed-captioning information embedded invideo stream 850 from content source 820. Social TV dongle 810 may thenanalyze the captured closed-captioning data in order to search for anyknown products. In some embodiments, the captured closed-captioninginformation may be compared to a database of known products. If anywords from the captured closed-captioning information match, forexample, product names in the database of known products, social TVdongle 810 may determine that the matched product is being shown or hasbeen recently shown on TV 830. In some embodiments, the capturedclosed-captioning information may be sent to social networking system160. Social networking system 160 may then search the database of knownproducts in order to identify specific products being shown on TV 830.

In certain embodiments, logo scraping may be utilized to determinespecific products placed in a program shown on TV 830. For example,social TV dongle 810 may analyze individual frames of video stream 850from content source 820 in order to identify logos of known products.Social TV dongle 810 may utilize any appropriate method such as imagecomparison to search for known logos. For example, captured image stillsof video stream 850 may be compared to logos in a database of knownproducts. If any portion of a captured image still matches a logo of aknown product, social TV dongle 810 may determine that the matchedproduct is being shown or has been recently been shown on TV 830. Insome embodiments, the captured image stills may be sent to socialnetworking system 160. Social networking system 160 may then search thedatabase of known product logos in order to identify specific productsbeing shown on TV 830.

In some embodiments, social TV dongle 810 may analyze audio from TV 830(e.g., from an HDMI audio stream or captured using a microphone) inorder to determine that a specific product has been mentioned, subjectto privacy settings. For example, if an actor says “Mega-Cola” in a showor advertisement, social TV dongle 810 may search a database of knownproducts for “Mega-Cola.” If the product is found in the database,social TV dongle 810 may determine that the matched product is beingmentioned or has been recently been mentioned on TV 830. In someembodiments, the captured audio may be sent to social networking system160. Social networking system 160 may then search the database of knownproducts in order to identify specific products mentioned on TV 830.

In some embodiments, OCR may be utilized to determine specific productsbeing shown on TV 830. For example, social TV dongle 810 may utilize OCRon captured image stills from video stream 850 in order to identifyspecific text being shown on the screen. Social TV dongle 810 may thenanalyze the captured text in order to identify known products. In someembodiments, the captured text may be compared to a database of knownproducts. If any words from the captured text match, for example,product names in the database of known products, social TV dongle 810may determine that the matched product is being shown or has beenrecently shown on TV 830. In some embodiments, the captured text may besent to social networking system 160. Social networking system 160 maythen search the database of known products in order to identify specificproducts being shown on TV 830.

If social TV dongle 810 or social networking system 160 determines thata specific product is being shown on TV 830 or has been mentioned on TV830, any appropriate action related to the specific product may betaken. In one example, mobile device 840 of the user may display awebsite 1910 associated with the product that is currently being shownor mentioned on TV 830. As a specific example for illustrative purposesonly, consider a TV show in which the product “Mega-Cola” is shown beingconsumed by an actor. Social TV dongle 810 may utilize any appropriatemethod such as those discussed above to determine the identity (e.g.,the specific brand or product name) of the displayed product. In thiscase, social TV dongle 810 determines that the specific product“Mega-Cola” has been shown on TV 830. Social TV dongle 810 may thencause a website 1910 associated with the specific product (i.e.,“Mega-Cola) being shown to be displayed on mobile device 840. In someembodiments, social TV dongle 810 may communicate directly with a mobileapp on mobile device 840 in order to display website 1910. In someembodiments, social networking system 160 may communicate with themobile app on mobile device 840 in order to display website 1910.

In some embodiments, mobile device 840 of the user may display a website1910 that allows viewers to purchase products that are currently beingshown or mentioned on TV 830. For example, social TV dongle 810 orsocial networking system 160 may determine that a specific product“Mega-Cola” has been shown on TV 830 either in an advertisement or in aprogram. Based on this determination, a website 1910 may be displayed onmobile device 840 that allows the user to purchase “Mega-Cola.” In someembodiments, the displayed website 1910 may be a website of a sponsorwho has purchased the right for the website of the sponsor to bedisplayed for certain products shown on TV 830.

In some embodiments, a mobile app 1920 associated with a specificproduct may be launched on mobile device 840 in response to adetermination that the specific product is or has been recently shown ormentioned on TV 830. For example, if social TV dongle 810 or socialnetworking system 160 determines that the specific product “Mega-Cola”is or has been recently shown being consumed by an actor in a TV show,instructions may be communicated to mobile device 840 to launch a mobileapp 1920 associated with “Mega-Cola.” As a result, the viewingexperience of user 101 may be enhanced because additional informationabout the specific products being shown on TV 830 may be automaticallypresented to the user.

FIG. 20 illustrates an example method 2000 for determining productsplaced in programs. The method may begin at step 2010, where dataassociated with a product shown on a display device is received. In someembodiments, the data is received by social networking system 160 fromsocial TV dongle 810. In some embodiments, the data may include dataobtained via OCR, logo-scraping, closed-captioning information, or anyother appropriate data.

At step 2020, a database of known products is accessed. In someembodiments, the database may include one or more of product names,logos, product images, and the like. At step 2030, the database of knownproducts is compared with the received data. Based on the comparison, anidentity of the product shown on the display device is determined instep 2040.

At step 2050, content related to the identified product shown on thedisplay may be presented to the user. In some embodiments, this contentmay be presented on a second screen of the user. For example, if it isdetermined in step 2040 that a specific brand of cola was advertised onthe display device, a website for the cola may appear on a mobile deviceof the user. As another example, a mobile app associated with thedetermined product may appear on the mobile device of the user.

Particular embodiments may repeat one or more steps of the method ofFIG. 20, where appropriate. Although this disclosure describes andillustrates particular steps of the method of FIG. 20 as occurring in aparticular order, this disclosure contemplates any suitable steps of themethod of FIG. 20 occurring in any suitable order. Moreover, althoughthis disclosure describes and illustrates particular components,devices, or systems carrying out particular steps of the method of FIG.20, this disclosure contemplates any suitable combination of anysuitable components, devices, or systems carrying out any suitable stepsof the method of FIG. 20.

FIG. 21 illustrates embodiments in which a second screen of a device ofuser 101 is utilized to present information related to what is currentlybeing displayed on TV 830. In some embodiments, the second screen ofuser 101 may be mobile device 840 such as a smartphone, a tabletcomputer, or any other appropriate display device. Some embodimentspresent a call-to-action 2110 on a second screen that is related tocontent being displayed on TV 830. In some embodiments, an API isprovided that permits content providers or advertisers to include audioin programming that triggers call-to-action 2110 or any other content onmobile device 840.

In some embodiments, call-to-action 2110 may be presented on a secondscreen of user 101 in response to what is currently being displayed onTV 830. In general, call-to-action 2110 may refer to a solicitedresponse in user 101. In some embodiments, call-to-action 2110 may be inthe form of a clickable button that performs a predefined action whenclicked. For example, as illustrated in FIG. 21, a singing competitionprogram on TV 830 may solicit user 101 to vote for contestants bydialing a telephone number. Concurrently with the display of thetelephone number, call-to-action 2110 may be presented on a secondscreen of user 101 such as mobile device 840. In this example,call-to-action 2110 is a button that appears on mobile device 840 thatallows user 101 to click the button to vote for contestant A withouthaving to dial the telephone number displayed on TV 830.

Call-to-action 2110, or any other content displayed on a second screenof user 101 such as mobile device 840, may be displayed in response toinstruction from social networking system 160 or social TV dongle 810.For example, social networking system 160 may send instructions tomobile device 840 either directly or via social TV dongle 810 to displaycall-to-action 2110. In certain embodiments, the instructions are sentto mobile device 840 based on metadata embedded in video stream 850. Forexample, metadata embedded within video stream 850 may indicate todisplay a certain call-to-action 2110 at a certain time in a program.Social TV dongle 810 or social networking system 160 may analyze themetadata and then send the instructions to mobile device 840 to displaycall-to-action 2110 at the appropriate time.

In some embodiments, call-to-action 2110, or any other content displayedon a second screen of user 101 such as mobile device 840, may bedisplayed in response to an advertisement on TV 830. For example, whenan advertisement for a specific product is being displayed on TV 830, acall-to-action 2110 for the specific product may be displayed on asecond screen of user 101 such as mobile device 840. As a specificexample, when an advertisement for a specific product is being displayedon TV 830, a call-to-action 2110 that says “CLICK HERE TO LEARN MOREABOUT THIS PRODUCT” may appear on mobile device 840. This call-to-action2110 may, for example, display a website or a mobile app for thespecific product being shown on TV 830 when pressed.

In certain embodiments, an API may be provided to content producers inorder to present information related to what is currently beingdisplayed on TV 830 to a second screen of user 101. For example, socialnetworking system 160 may provide an API to content producers thatdisplays call-to-action 2110 based on certain audio or video beingdisplayed on TV 830. To illustrate this, consider the TV showillustrated in FIG. 21. In this show, the content provider desires thatcall-to-action 2110 appear on mobile device 840 when the telephonenumber appears on the screen of TV 830. To accomplish this, someembodiments of mobile device 840 or social TV dongle 810 may listen fora specific tone or audio emitted from TV 830. When the correct sound isdetected, call-to-action 2110 may be displayed.

In certain embodiments, content may be displayed on a second screen ofuser 101 in order to provide a simulcast viewing experience for user101. For example, consider a scenario where user 101 is watching asporting event on TV 830. Concurrent with viewing the sporting event,social networking system 160 may cause sports scores to be displayed ona second screen of user 101 such as mobile device 840. In someembodiments, a scoreboard of the current game being watched may bedisplayed on the second screen. For example, if user 101 is viewing abaseball game on TV 830, a box score of the current baseball game (e.g.,current score, pitch count, etc.) may be displayed on mobile device 840.In some embodiments, sports scores of other similar games to what iscurrently being viewed may be displayed on the second screen of user101. For example, if user 101 is viewing a football game on TV 830,scores of other football games may be displayed on mobile device 840.

In some embodiments, TV 830 may be utilized as a second screen formobile device 840 of user 101. For example, user 101 may viewprogramming on mobile device 840 from OTT provider 403 (e.g., Netflix).While user 101 is viewing content on mobile device 840 from OTT provider403, social area 1140 may be displayed on TV 830. For example, chatsessions or other messages from social networking system 160 that arerelated to the current program may be displayed in social area 1140 onTV 830. As another example, a web browser of social networking system160 may be utilized on mobile device 840. While the user is utilizingthe web browser on mobile device 840, social area 1140 may be displayedon TV 830.

FIG. 22 illustrates an example method 2200 for utilizing a second screento present information related to what is currently being displayed on aTV. The method may begin at step 2210, where it is determined that auser is viewing content on a first display device. For example, it maybe determined that the user is viewing a show on TV 830. In someembodiments, any appropriate method may be determined that a user isviewing content on the first display device. For example, social TVdongle 810 may utilize BLUETOOTH, WiFi, acoustic fingerprinting, GPSdata, electrical interference, closed-captioning, explicit signals fromthe user, HDMI signal, or any other appropriate method to determine thatthe user is viewing the content. In some embodiments, the display devicemay be a TV, a smartphone, a computer, a tablet computer, and the like.

At step 2220, a second display device of the user is determined. Forexample, social TV dongle 810 may determine that the user is interactingwith a smartphone while viewing the content on the first display device.As another example, social TV dongle 810 may determine that the user isinteracting with a tablet computer while viewing the content on thefirst display device.

At step 2230, one or more actions related to the content being viewed onthe first display device are determined. For example, it may bedetermined that certain text has been displayed on the first displaydevice. As another example, it may be determined that a certain soundhas been emitted from display device. As yet another example, it may bedetermined that certain metadata is embedded in a video stream.

At step 2240, a call-to-action is displayed on the second displaydevice. In some embodiments, the call-to-action solicits a response fromthe user related to the determined one or more actions. For example, ifit is determined in step 2230 that a user is invited to vote for aspecific contestant in a talent competition program, a call-to-actionvoting button may appear on the second display device. Thecall-to-action voting button may cause an indication to be sent tosocial networking system 160 or any other location where the user's votemay be tallied.

Particular embodiments may repeat one or more steps of the method ofFIG. 22, where appropriate. Although this disclosure describes andillustrates particular steps of the method of FIG. 22 as occurring in aparticular order, this disclosure contemplates any suitable steps of themethod of FIG. 22 occurring in any suitable order. Moreover, althoughthis disclosure describes and illustrates particular components,devices, or systems carrying out particular steps of the method of FIG.22, this disclosure contemplates any suitable combination of anysuitable components, devices, or systems carrying out any suitable stepsof the method of FIG. 22.

FIG. 23 illustrates embodiments in which a second screen of a device ofuser 101 is utilized to allow user 101 to participate in chat sessionsrelated to what is currently being displayed on TV 830. In someembodiments, the second screen of user 101 may be mobile device 840 suchas a smartphone, a tablet computer, or any other appropriate displaydevice. In general, when user 101 selects a show to watch on TV 830, theshow may be displayed on TV 830 and a mobile chat app 2320 may beautomatically launched on mobile device 840 of user 101 that provides adedicated chat experience for the show being viewed. This may enableuser 101 to view chat messages from other users such as friends 101 a-ewho are currently watching the show or have watched the show in thepast. In addition, user 101 may be able to post chat messages about theshow. The chat messages may be time-stamped to allow the messages toappear to other users during the appropriate time during the show evenif the show is being viewed in time-delay (e.g., from a DVR).

Mobile chat app 2320 may provide various chat experiences for user 101.In some embodiments, mobile chat app 2320 may include a general lobbyfor the show currently being viewed. For example, if the show beingviewed is one of multiple episodes of a particular series, mobile chatapp 2320 may include a general lobby for the particular series. In someembodiments, mobile chat app 2320 may include chat rooms for specificseasons or episodes of a particular series. Some embodiments may provideprivate chat rooms in which users 101 may conduct private conversationsabout the current show that are not posted to social networking system160.

In operation, when user 101 selects a program to view on TV 830, mobilechat app 2320 may be automatically launched on mobile device 840.Alternatively, user 101 may manually launch mobile chat app 2320 onmobile device 840. In some embodiments, user 101 may be presented withoptions to join various chat rooms such as a general lobby for theparticular series, chat rooms for specific seasons or episodes of aparticular series, and private chat rooms. Once user 101 has selected achat room, one portion of mobile chat app 2320 may begin displaying chatmessages about the show currently being viewed on TV 830. In someembodiments, chat messages from both friends 101 a-e and other users whoare not friends of user 101 may be displayed in chat app 2320. The chatmessages displayed in mobile chat app 2320 may automatically update asusers post chat message about the current show. As a result, user 101may be able to quickly jump into a chat environment with friends andstrangers about a specific show user 101 is currently viewing on TV 830.

In certain embodiments, a portion of chat app 2320 may provide an areathat allows user 101 to create chat messages. As illustrated in FIG. 23,for example, a lower portion of mobile chat app 2320 may provide a textinput area that allows user 101 to enter text using mobile device 840 inorder to create a chat message about the current show being viewed on TV830. In some embodiments, voice recognition may be utilized to entertext into mobile chat app 2320. In one example, a microphone of mobiledevice 840 may be utilized to capture spoken chat messages by one ormore users 101. Mobile device 840 may then convert the spoken chatmessages into text that is displayed in the text input area of mobilechat app 2320. User 101 may then edit or approve the chat message forposting to social networking system 160. In another example, voicerecognition may be utilized to automatically add a conversation takingplace in a room to a chat session (with our without the use of a secondscreen of user 101 such as mobile device 840). For example, a microphonecoupled to social TV dongle 810 or a microphone of mobile device 840 maycapture a conversation between two or more users watching a show on TV830. This conversation may be analyzed by social TV dongle 810 or sentto social networking system 160 for analysis using voice recognition.The conversation may be converted to chat messages and automaticallyposted to a chat room associated with the show currently being viewed onTV 830. In some embodiments, the conversation may be determined bysubtracting audio of what the user is currently watching from thecaptured audio.

In certain embodiments, chat message displayed in mobile chat app 2320on mobile device 840 are concurrently shown on TV 830 in a chat area2310. In some embodiments, chat area 2310 may be social area 1140described above. Chat area 2310 may be any shape, size, or transparencyand may be located in any location on the screen of TV 830. In someembodiments, chat area 2310 may be placed on TV 830 to avoid importantareas of the screen, as discussed above. In certain embodiments, theshape, size, or location of chat area 2310 may be dynamically altered,as discussed above. For example, chat area 2310 may be located to avoidoverlapping faces, static portions of the screen such as a scoreboard,or any other determined important area of the screen. In someembodiments, social networking system 160 may send chat messages tosocial TV dongle 810 for display in chat area 2310.

The chat messages that are displayed in chat area 2310 on TV 830 may insome embodiments include all the chat messages that are displayed inchat app 2320. However, some embodiments may filter the chat messages toonly display chat messages that are important to user 101 in chat area2310 on TV 830. For example, FIG. 23 illustrates two chat messagedisplayed in chat app 2320 of user 101—one chat message from Friend Aand one chat message from User B. Friend A may refer to friends 101 a-eof user 101 and User B may refer to any other user of social networkingsystem 160 who does not have a connection in social graph 300 to user101 (e.g., is not a friend of user 101). Because user 101 may not wantto view every chat message on TV 830 about the current program beingviewed, social networking system 160 may cause only chat messages offriends 101 a-e of user 101 to be displayed in chat area 2310 on TV 830.For example, as illustrated in FIG. 23, only the chat message fromFriend A of user 101 is displayed in chat area 2310 on TV 830 of user101. Social networking system 160 may determine what chat message todisplay in chat area 2310 on TV 830 using preferences from user 101(e.g., user 101 indicates only to display chat messages of friend,friends of friends, chat messages according to likes, etc.), connectionsto user 101 in social graph 300, or any other appropriate method.

In some embodiments, chat messages associated with programs being viewedon TV 830 may be time-stamped to allow the chat messages to be syncedwith the program and later viewed at the same point within the programas when the chat messages were originally posted. For example, manyusers utilize a DVR to view recorded programs at a later time from whenthey were originally broadcast. To allow users who are viewing recordedor time-shifted content to also view chat messages related to thecontent, some embodiments time-stamp chat messages and store the chatmessages to be displayed at a later time when the content is beingviewed. For example, social networking system 160 may store time-stampedchat messages in a data repository such as a database. The chat messagesmay be time-stamped with, for example, an absolute time (e.g., Feb. 1,2010 at 23:12) or a time relative to the content associated with thechat message (e.g., 0:16 after the beginning of the program). Inaddition, some embodiments may store other metadata with the chatmessages such as an identification of the user who posted the chatmessage, an identification of the content associated with the chatmessage, and the like. Then, when users view the content at a latertime, the stored chat messages related to the content may be retrievedand displayed to the users at the appropriate time in the content. Forexample, if a particular friend of a user made the comment “THIS ROCKS!”during a point of a show, that comment may appear at that same point ofthe show to users viewing the show at a later time. The comments mayappear on mobile device 840, TV 830, or both devices concurrently. As aresult, users may be able to view relevant chat messages about thecontent currently being viewed even if the content is being viewed at alater time from when the content originally aired.

In some embodiments, chat messages related to content being viewed on TV830 may be organized by social networking system 160 for presentation tousers 101 of social networking system 160. For example, socialnetworking system 160 may organize all chat messages about a particularTV show into comment threads based on the content. In some embodiments,this may include comment threads for particular shows, particularepisodes, particular seasons, etc. These comment threads may then beviewed by users 101 using, for example, a web browser. In certainembodiments, social networking system 160 may dynamically form groups ofusers 101 based on detected shared content experience. For example, ifsocial networking system 160 detects that certain users 101 have allviewed a particular TV show, social networking system 160 may recommendthat each of the users join a group on social networking system 160 forthat particular show.

In some embodiments, user 101 may participate in a video chat about thecurrent show being viewed. For example, webcam 970 coupled to social TVdongle 810 may capture video of user 101 as he is watching a particularshow. At the same time, other webcams 970 may capture video of otherusers watching the particular show. Social networking system 160 maythen display the videos of each user in small boxes within chat area2310 on TV 830. For example, social networking system 160 may displaythe videos of a certain number of friends of user 101 who are alsocurrently watching the particular show. In some embodiments, facedetection may be utilized to only display the face or head of users 101within chat area 2310. In certain embodiments, gestures of user 101 forvideo chats may be captured and analyzed as discussed above. Forexample, user 101 may perform a “shhhh” gesture in order to mute one ormore other users being displayed in the video boxes within chat area2310 on TV 830. Social TV dongle 810 may capture and determine thisgesture by user 101 and mute the appropriate video of the user withinchat area 2310 that user 101 indicates to mute.

In some embodiments, an indication of second screen augmentation ispresented to user 101. For example, an indication such as an icon may bepresented to user 101 to indicate that a certain program is availablefor second-screen interaction (e.g., the user may chat about the programon a mobile device while watching the program on a TV). In someembodiments, the indication may be shown in a newsfeed or a list view ofavailable programs. In some embodiments, the indication may be presentedeither before or after user 101 selects to tune to a particular program.

FIG. 24 illustrates an example method 2400 for utilizing a second screento participate in a chat session related to what is currently beingdisplayed on a TV. The method may begin at step 2410, where it isdetermined that a user is viewing particular content on a first displaydevice. For example, it may be determined that the user is viewing aparticular show on TV 830.

At step 2420, a second display device of the user is determined. In someembodiments, the second display device is a mobile device of the user.In some embodiments, the second display device is TV 830.

At step 2430, a chat room is displayed on the second display device. Thechat room may include various rooms such as a general lobby for aparticular series of a show. In some embodiments, the chat room mayinclude chat rooms for specific seasons or episodes of a particularseries. Some embodiments may provide private chat rooms in which usersmay conduct private conversations about the current show.

At step 2440, chat messages related to the particular content aredisplayed in the chat room on the second display device. The chatmessage may be determined from a social graph such as social graph 300.In some embodiments, chat messages from friends and non-friends of theuser are displayed in the chat room. In certain embodiments, only chatmessage from friends are displayed in the chat room. In someembodiments, the user may specify preferences regarding what chatmessages to display. In certain embodiments, important chat message aredisplayed on both the first and second display devices.

Particular embodiments may repeat one or more steps of the method ofFIG. 24, where appropriate. Although this disclosure describes andillustrates particular steps of the method of FIG. 24 as occurring in aparticular order, this disclosure contemplates any suitable steps of themethod of FIG. 24 occurring in any suitable order. Moreover, althoughthis disclosure describes and illustrates particular components,devices, or systems carrying out particular steps of the method of FIG.24, this disclosure contemplates any suitable combination of anysuitable components, devices, or systems carrying out any suitable stepsof the method of FIG. 24.

FIG. 25 illustrates embodiments in which a playlist 2510 for content tobe viewed on TV 830 is generated and presented to one or more users 101.In some embodiments, playlist 2510 is a shared playlist that combinesinterests of two or more users. In general, playlist 2510 is generatedusing users' interests as determined by social networking system 160using social graph 300. In some embodiments, playlist 2510 is generatedby MSO 402 using information obtained from social networking system 160about one or more users 101. In some embodiments, playlist 2510 isgenerated by social networking system 160. Playlist 2510 may bedisplayed on mobile device 840, TV 830 (e.g., as a programming guide),or any other appropriate device. Playlist 2510 may allow multiple users101 to view content that is relevant to all of their interests. As aresult, the social experience of the users may be increased.

In certain embodiments, playlist 2510 is a list of one or more movies,TV shows, or other content to be viewed on TV 830. The content ofplaylist 2510 may be determined based on the interests of two or moreusers. For example, consider a couple trying to decide what to watchtogether on TV 830. The couple may seek a recommendation using, forexample, mobile device 840 or remote control device 950. In someembodiments, MSO 402 may detect the requested recommendation and send aquery to social networking system 160 for information about the users.In some embodiments, social networking system 160 may detect therequested recommendation using, for example, social TV dongle 810 andproceed to query social graph 300 for information about the users. Theinformation may include, for example, what the users are doing, posts ofthe users, likes of the users, mutual friends of the users, posts orlikes by mutual friends of the users, programming guides of mutualfriends of the users, etc. This information may then be used todetermine content to include in playlist 2510. As a specific example, ifthe users and their mutual friends have recently discussed the topic“weddings” on social networking system 160, then playlist 2510 mayinclude TV shows or movies about weddings.

In certain embodiments, social TV dongle 810 may determine who isviewing TV 830 in order to determine content to provide in playlist2510. For example, social TV dongle 810 may utilize any method discussedabove to determine who is in close proximity to TV 830 or social TVdongle 810. These methods may include utilizing BLUETOOTH to gatherinformation from multiple mobile devices 840 as discussed above,utilizing webcam 970 and facial recognition technology to detect users'faces in the room, using voice recognition to detect users' voices, etc.Once social TV dongle 810 or social networking system 160 determines whois in the room, social graph 300 may be queried to determine theinterests of all the determined users. The interests of all thedetermined users may then be combined to determine the content forplaylist 2510. Particular embodiments of determining shared interestsare discussed further below.

In certain embodiments, social networking system 160 may utilize socialcoefficients to determine content for playlist 2510. For example, socialnetworking system 160 may first query social graph 300 to determinesocial coefficients of programs for an individual user 101. In someembodiments, this may involve determining social coefficients of allavailable content in social networking system 160. Some embodiments mayutilize coefficients as described in U.S. patent application Ser. No.12/978,265 entitled “Contextually Relevant Affinity Prediction in aSocial Networking System,” which is incorporated herein by reference.Social networking system 160 may utilize the following information fromsocial graph 300 in determining coefficients of specific programs forindividual user 101:

-   -   the number of friends 101 a-e of user 101 who have liked the        specific program    -   the total number of users of social networking system 160 who        have liked the specific program    -   the number of friends 101 a-e of user 101 who are currently        watching the specific program    -   the total number of users of social networking system 160 who        are currently watching the specific program    -   a taste algorithm (e.g., users who liked the specific program        will also like this other program.)        Social networking system 160 may then combine the information        above in order to determine a coefficient associated with user        101 for each program. For example, consider a scenario in which        a query of social graph 300 for the specific program yielded the        following results:    -   the number of friends 101 a-e of user 101 who have liked the        specific program=5    -   the total number of users of social networking system 160 who        have liked the specific program=1000    -   the number of friends 101 a-e of user 101 who are currently        watching the specific program=2    -   the total number of users of social networking system 160 who        are currently watching the specific program=250        The combination of these example results would yield a        coefficient of 5+1000+2+250=1257 for this program for user 101.        Social networking system 160 then performs similar calculations        in order to determine coefficients of other programs for user        101. In certain embodiments, the resulting coefficients of all        shows are then ranked for user 101. A predetermined number of        the top-ranked shows may then be presented to user 101 in        playlist 2510.

In embodiments where it is determined that multiple users 101 areviewing TV 830, social networking system 160 may first determinespecific programs to recommend to each individual user 101 as discussedabove (e.g., using social coefficients) and then perform groupaggregation in order to determine recommendations for the group of users101. For example, consider a scenario where three users 101 are seekingrecommendations on what to watch on TV 830. Social networking system 160may first determine social coefficients as discussed above for a firstshow for each of the three users. Social networking system 160 may thenaverage the three coefficients for the first show to determine anaveraged coefficient for the first show. Social networking system 160may then perform similar computations for other shows to determineaveraged coefficients for each show. The average coefficients may thenbe ranked for the three users and a predetermined number of thetop-ranked shows (or shows that have a coefficient greater than apredetermined amount) may then be presented in a shared playlist 2510for the three users. As a specific example, the following coefficientsmay be determined for shows A, B, and C for the three users:

-   -   User 1: show A coefficient=30; show B coefficient=50, show C        coefficient=2    -   User 2: show A coefficient=100; show B coefficient=40, show C        coefficient=1    -   User 3: show A coefficient=200; show B coefficient=300, show C        coefficient=3        The coefficients above may then be averaged to determine average        coefficients as follows:    -   Show A average coefficient=(30+100+300)/3=110    -   Show B average coefficient=(50+40+300)/3=130    -   Show C average coefficient=(2+1+3)/3=2        The average coefficients may then be ranked for the three users        and a predetermined number of the top-ranked shows (or shows        that have a coefficient greater than a predetermined amount) may        then be presented in a shared playlist 2510 for the three users.        In the above example, Show B may be listed first in playlist        2510, followed by Show A, and then Show C.

In some embodiments, weighting factors may be used in determiningaverage coefficients of a group of users. For example, consider a familythat includes a father, a mother, and a child. Social networking system160 may apply weights to the coefficients of the father, mother, andchild in calculating the average coefficients above. For example, ifsocial networking system 160 determines that the father watches the mostTV shows, social networking system 160 may apply a higher weight to hiscoefficients than the mother or child. Or, if social networking system160 determines that most of the TV shows watched in the house arechildren's shows, it may apply a higher weight to the child'scoefficients than the mother or father. Thus, recommendations inplaylist 2510 may be altered to prioritize content for the person whoconsumes the most TV shows.

In some embodiments, social networking system 160 finds the union ofindividual show recommendations when determining what content to displayin a shared playlist 2510. For example, if two users are seekingrecommendations on what to watch on TV 830, social networking system 160may first determine shows to recommend for each individual user asdiscussed above. Social networking system 160 may then determine whetherthere are any common recommended shows for the two users. For example,if Shows A and B are recommended to a first user and Shows A and C arerecommended to a second user, Show A may be included in a sharedplaylist 2510 for the two users, since it was recommended to both users.

As discussed above, a taste algorithm may be utilized by socialnetworking system 160 to determine recommendations to include inplaylist 2510. For example, social networking system 160 may firstdetermine first programs that user 101 watched, liked, or otherwiseinteracted with on social networking system 160. In some embodiments,the first programs may be determined using social graph 300 or social TVdongle 810. In some embodiments, the programs may be any of live TV,programming from OTT provider 403 such as Netflix, programs from a DVR,on-demand programming, and the like. Once the first programs have beendetermined, social networking system 160 may then search social graph300 to determine other users or friends 101 a-e of user 101 who alsowatched, liked, or otherwise interacted with the first program on socialnetworking system 160. Social networking system 160 may then searchsocial graph 300 for second programs that the determined users orfriends watched, liked, or otherwise interacted with on socialnetworking system 160. The determined second programs may then berecommended to user 101 in playlist 2510 as discussed above. In someembodiments, weighting factors may be utilized in determining what showsto recommend to a user. For example, second programs viewed by a user'sfriends may have a higher weight in determining recommendations for auser than second programs viewed by users who are not friends.

As an example for illustrative purposes only, consider a scenario wherea first user has liked the program “Garage Talk” in the past. Socialnetworking system 160 may query social graph 300 in order to determineother users or friends of the first user who have also liked “GarageTalk.” For example, social networking system 160 may determine thatFriend A and User B also liked “Garage Talk.” Social networking system160 may then query social graph 300 for other shows liked by the otherusers or friends who also liked “Garage Talk.” For example, socialnetworking system 160 may determine that Friend A and User B, who liked“Garage Talk,” also liked the shows “Shop Journal” and “Cars Galore.”Social networking system 160 may then recommend “Shop Journal” and “CarsGalore” to the first user. If, for example, Friend A liked “ShopJournal” and User B (who is not a friend of the first user) liked “CarsGalore,” then “Shop Journal” may appear higher in playlist 2510 than“Shop Journal.”

In some embodiments, social networking system 160 may cross-referencethe actors or the people who were involved in making a particularprogram in determining what shows to recommend to user 101 in playlist2510. For example, social networking system 160 may access an externaldatabase such as the Internet Movie Database (IMDb) in order todetermine the actors, producers, writers, etc. of a particular showliked or watched by user 101 or the connections of user 101. Socialnetworking system 160 may then cross-reference the determinedinformation with other shows in order to determine other shows that theactors, producers, writers, etc. were also involved with. As an examplefor illustrative purposes only, consider a scenario where friend 101 aof user 101 likes Show A that features Actors A and B. Social networkingsystem 160 may recommend Show A to user 101 if it determines that user101 likes Actor B. That is, because both user 101 and friend 101 a likeactors that are in a particular show, social networking system 160 mayrecommend that user 101 and friend 101 a watch the same show.

In some embodiments, other data points may be used to infer how much auser liked a particular show. For example, social networking system 160may determine how much of a particular show a user watched beforechanging the channel, turning off TV 830, or any other action thatindicates disinterest in a TV show. Some embodiments may determine thisusing, for example, social TV dongle 810 discussed above. In certainembodiments, social networking system 160 may determine that a userbegan viewing a show and then checked in at another location before theend of the show. In certain embodiments, this may indicate that the userbecame disinterested in the show. In some embodiments, social networkingsystem 160 may determine where the user checked before the end of theshow. If, for example, the user stopped watching a sporting event andthen checked in at a sports bar, social networking system 160 maydetermine that the user did not become disinterested in the sportingevent but instead went to a location to continue watching the sportingevent with others. Social networking system 160 may utilize posts tosocial networking system 160, GPS data from mobile device 840, or anyother appropriate data to determine whether the user becamedisinterested in a particular show.

In certain embodiments, social networking system 160 may determinetrending shows to recommend to user 101 in playlist 2510. For example,social networking system 160 may analyze posts, likes, or any otherinteraction with social networking system 160 in order to determineshows that are trending. Shows that are trending may be, for example,shows that have received the most likes within a predetermined period.Social networking system 160 may then include one or more trending showsin playlist 2510.

In some embodiments, users 101 may be notified when a particular programis viewed by a large percentage of their friends. For example, if overhalf of friends 101 a-e of user 101 are currently viewing or plan toview a particular program, a notification of the program may be sent touser 101 via social networking system 160. In some embodiments, socialnetworking system 160 may determine that a large percentage of a user'sfriends are watching or are planning to watch a program using one ormore of the coefficients discussed above, close friends of the user(i.e., using social graph 300), similar tastes, and the like. In someembodiments, the notification may include a link or button that allowsuser 101 to press to begin watching the recommended show.

In some embodiments, social networking system 160 may provide a “bestfor you” recommendation to user 101 in playlist 2510. The “best for you”recommendation may be the program chosen from the currently availableprograms that user 101 is most likely to watch. In some embodiments,social networking system 160 may first determine programs that arecurrently available to watch (e.g., programs that are currently airing,programs that are available via pay-per-view, programs that are recordedon a DVR, programs available from services such as Hulu and Netflix,etc.). Social networking system 160 may then perform any appropriatemethod to rank the available programs according to what user 101 islikely to want to watch. In some embodiments, this may include usingsocial coefficients described above. Social networking system 160 maythen choose the top-ranked program and present an option to user 101 toview the program. In certain embodiments, the top-ranked program may bepresented to user 101 along with an indication such as “best for you” orany other appropriate indication.

FIG. 26 illustrates an example method 2600 for generating an exampleshared playlist. The method may begin at step 2610, where identities ofa first and second user are determined. This may be accomplished by, forexample, analyzing MAC addresses of the users' mobile devices.

At step 2620, a first plurality of recommended viewing content for thefirst user is determined. In certain embodiments, social graph 300 isutilized to determine the first plurality of recommended viewingcontent. In some embodiments, social coefficients may be used todetermine the first plurality of recommended viewing content.

At step 2630, a second plurality of recommended viewing content for thesecond user is determined. In certain embodiments, social graph 300 isutilized to determine the second plurality of recommended viewingcontent. In some embodiments, social coefficients may be used todetermine the second plurality of recommended viewing content.

At step 2640, a combined plurality of recommended viewing content isdetermined using the first plurality of recommended viewing content andthe second plurality of recommended viewing content. In someembodiments, social coefficients may be used to determine the combinedplurality of recommended viewing content. In some embodiments, thecombined plurality of recommended viewing content may be the union ofthe first and second plurality of recommended viewing content.

At step 2650, the combined plurality of recommended viewing content ispresented to the first and second users. In some embodiments, thecombined plurality of recommended viewing content is displayed on amobile device of one or both of the first and second users. In certainembodiments, the combined plurality of recommended viewing content isdisplayed on a TV that both users are viewing.

Particular embodiments may repeat one or more steps of the method ofFIG. 26, where appropriate. Although this disclosure describes andillustrates particular steps of the method of FIG. 26 as occurring in aparticular order, this disclosure contemplates any suitable steps of themethod of FIG. 26 occurring in any suitable order. Moreover, althoughthis disclosure describes and illustrates particular components,devices, or systems carrying out particular steps of the method of FIG.26, this disclosure contemplates any suitable combination of anysuitable components, devices, or systems carrying out any suitable stepsof the method of FIG. 26.

FIG. 27 illustrates embodiments in which time-sensitive buttons 2710 maybe displayed on mobile device 840 in response to what is being shown onTV 830. In general, time-sensitive buttons 2710 allow user 110 to postcontent to social networking system 160 that is synced with what isbeing displayed on TV 830. Time-sensitive buttons 2710 may include, forexample, a like button 2710 a and a share button 2710 b as discussedbelow. User 101 may press or otherwise interact to select buttons 2710when viewing content on TV 830. Mobile device 840 communicates anindication of the interaction with buttons 2710 to social networkingsystem 160 either directly or via social TV dongle 810.

In certain embodiments, time-sensitive button 2710 may be a like button2710 a that allows user 101 to indicate to social networking system 160that he likes content being shown on TV 830. For example, user 101 maypress like button 2710 a during a particular chase scene of a show “EastStreet” being viewed on TV 830. An indication of the user pressing likebutton 2710 a during the chase scene may be sent to social networkingsystem 160 where a post such as “User 101 likes the chase scene in EastStreet” may be posted to the wall of user 101. As another example, user101 may press like button 2710 a that appears on mobile device 840during a commercial or advertisement for a specific product such as“Mega-Cola.” An indication of the user pressing like button 2710 aduring the advertisement may be sent to social networking system 160where a post such as “User 101 likes Mega-Cola” may be posted to thewall of user 101. In certain embodiments, the indication of the userpressing like button 2710 a may cause an edge to be created in socialgraph 300. For example, if user 101 presses like button 2710 a during acommercial or advertisement for “Mega-Cola,” an edge 306 may be createdbetween user node 302 for user 101 and a concept node 304 (e.g., aproduct page) for “Mega-Cola.”

In some embodiments, like button 2710 a may be used to indicate tosocial networking system 160 that user 101 likes any granularity ofcontent on TV 830. For example, like button 2710 a may be used toindicate that user 101 likes a particular channel, content provider,show, season, episode, scene, actor, product shown in a program, productshown in an advertisement or commercial, piece of music, or any othercontent shown on TV 830. In certain embodiments, like button 2710 a mayautomatically appear on mobile device 840 while user 101 is watchingcontent on TV 830. In some embodiments, like button 2710 a may bemanually displayed on mobile device 840 by user 101 by, for example,launching a mobile app on mobile device 840.

In certain embodiments, time-sensitive button 2710 may be a share button2710 b that allows user 101 to share content being shown on TV 830 withother users of social networking system 160. For example, user 101 maypress share button 2710 b during a particular play of a sporting eventbeing viewed on TV 830. An indication of the user pressing share button2710 b during the particular play may be sent to social networkingsystem 160 where a post that includes a clip or an image of theparticular play may be posted to the wall of user 101. As anotherexample, user 101 may press share button 2710 b on mobile device 840during a particular conversation occurring on TV 830. An indication ofthe user pressing share button 2710 b during the conversation may besent to social networking system 160 where a post that includes the textof the conversation (e.g., from subtitles or closed-captioning) may beposted to the wall of user 101.

In some embodiments, share button 2710 b may be used to share anygranularity of content shown on TV 830 with other users of socialnetworking system 160. For example, share button 2710 b may be used toshare images, clips (e.g., short videos), text, music, or any othercontent shown on TV 830 with users of social networking system 160. Incertain embodiments, share button 2710 b may automatically appear onmobile device 840 while user 101 is watching content on TV 830. In someembodiments, share button 2710 b may be manually displayed on mobiledevice 840 by user 101 by, for example, launching a mobile app on mobiledevice 840.

In some embodiments, a mobile app of social networking system 160running on mobile device 840 may include an area that allows user 101 toexplicitly share content currently being viewed. For example, the mobileapp may include a text entry area in which the user can type a messagesuch as “I'm watching Lone Survivor.” In some embodiments, the textentry area may autocomplete a user's message. For example, if the usertypes “I'm watching,” the mobile app may query the user's contentproviders (e.g., MSO 402) in order to determine what the user iscurrently watching. The mobile app may then autocomplete the user'sphrase using the name of the program currently being viewed.

FIG. 28 illustrates an example method 2800 for displaying time-sensitivebuttons in response to what is being shown on a TV. The method may beginat step 2810, where it is determined that a user is viewing content on afirst display device. For example, it may be determined that the user isviewing a particular show on TV 830.

At step 2820, a time-sensitive button is displayed on a second displaydevice of the user. In some embodiments, the time-sensitive button is alike button. In some embodiments, the time-sensitive button is a sharebutton. The time-sensitive button may appear automatically (e.g., fromsocial networking system 160), or the user may manually launch thetime-sensitive button using, for example, a mobile app on a mobiledevice.

At step 2830, an indication that the user has selected thetime-sensitive button is received. In some embodiments, it is receivedat social networking system 160. In certain embodiments, the indicationcorresponds to the user pressing the time-sensitive button.

At step 2840, one or more edges are created in a social graph based onthe viewed content and the received indication. For example, if the userpresses the time-sensitive like button during a commercial for aspecific product, social networking system 160 may add a “like” edgebetween nodes for the user and the specific product. As another example,if the user presses the time-sensitive share button during a movie,social networking system 160 may add a “recommend” edge between nodesfor the user and the movie in the social graph.

Particular embodiments may repeat one or more steps of the method ofFIG. 28, where appropriate. Although this disclosure describes andillustrates particular steps of the method of FIG. 28 as occurring in aparticular order, this disclosure contemplates any suitable steps of themethod of FIG. 28 occurring in any suitable order. Moreover, althoughthis disclosure describes and illustrates particular components,devices, or systems carrying out particular steps of the method of FIG.28, this disclosure contemplates any suitable combination of anysuitable components, devices, or systems carrying out any suitable stepsof the method of FIG. 28.

FIG. 29 illustrates embodiments in which devices may be utilized assmart remote controls for TV 830. In some embodiments, mobile device 840may display a channel switching mobile app 2910 that a user may interactwith to control what is being viewed on TV 830. In some embodiments,remote control device 950 may be utilized to interact with social TVdongle 810 in order to control what is being viewed on TV 830. Social TVdongle 810 may control what is being viewed on TV 830 by sendingcommands via IR to content source 820. In some embodiments, social TVdongle 810 may be the only source of content for TV 830 and may receivecontent to display from social networking system 160. In someembodiments, channel switching mobile app 2910 may automatically appearwhen user 101 comes within close proximity to social TV dongle 810. Incertain embodiments, channel switching mobile app 2910 replaces a lockscreen of mobile device 840.

In some embodiments, user 101 may interact with channel switching mobileapp 2910 on mobile device 840 to control what is viewed on TV 830. Forexample, user 101 may launch channel switching mobile app 2910 thatdisplays available content to watch on TV 830. The available content maybe, for example, a social programming guide (SPG) discussed below inreference to FIG. 31, recommendations as discussed above in reference toFIG. 25, or any other available content from social networking systemsocial networking system 160, MSO 402, or OTT provider 403. In someembodiments, channel switching mobile app 2910 may display one or morefacepiles (e.g., multiple photos of users of social networking system160). In some embodiments, social content is provided in channelswitching mobile app 2910. As illustrated in FIG. 29, for example,social data from social networking system 160 may be included next toshows that user 101 may select to watch. The social data may include,for example, friends or connections of user 101 who are currentlywatching the show, friends or connections of user 101 who like the show,friends or connection of user 101 who are planning to watch the show, orany other available social data for the show from social networkingsystem 160. In addition, channel switching mobile app 2910 may include abutton or link for each show that a user may press or otherwise interactwith to begin watching the show. Once pressed, mobile device 840 maysend an indication of the interaction to social TV dongle 810 or socialnetworking system 160. Social TV dongle 810 may subsequently send IRcommands to content source 820 in order to, for example, tune contentsource 820 to the appropriate channel for the selected show.

In some embodiments, user 101 may utilize remote control device 950 inorder to interact with social TV dongle 810 and thus control what isviewed on TV 830. For example, a dedicated button may be included onremote control device 950 that allows control of social TV dongle 810.In some embodiments, the dedicated button may launch a user interface onTV 830 from social TV dongle 810 for controlling functions of TV 830.User 101 may then use general controls of remote control device 950 tocontrol function of social TV dongle 810 or TV 830. Social TV dongle 810may subsequently send IR commands to content source 820 in order to, forexample, tune content source 820 to the appropriate channel for aselected show.

FIG. 30 illustrates an example method 3000 for utilizing devices assmart remote controls for a TV. The method may begin at step 3010 whereit is determined that a user is viewing particular content on a firstdisplay device. For example, it may be determined that the user isviewing a particular show on TV 830.

At step 3020, an interface is displayed on a second display device forcontrolling content displayed on the first display device. In someembodiments, the interface may include shows or movies that the user mayselect to watch on the first display device. In certain embodiments, theinterface may be displayed on a mobile device of the user.

At step 3030, an indication that the user has selected an option fromthe interface on the second display device is received. In someembodiments, the indication is received at social networking system 160.In certain embodiments, the indication corresponds to the user pressinga button, link, or icon on the interface.

At step 3040, instructions are sent to the first display device based onthe selected option. In some embodiments, the instructions are sent tothe first display device via social TV dongle 810. For example, theinstructions may be to display a certain show or movie. In certainembodiments, social TV dongle 810 may send instructions via, forexample, IR to content source 820.

At step 3050, one or more edges in a social graph are created based onthe selected option. For example, if the user presses an icon to beginviewing a particular program, social networking system 160 may add a“watched” edge between nodes for the user and the particular program.

Particular embodiments may repeat one or more steps of the method ofFIG. 30, where appropriate. Although this disclosure describes andillustrates particular steps of the method of FIG. 30 as occurring in aparticular order, this disclosure contemplates any suitable steps of themethod of FIG. 30 occurring in any suitable order. Moreover, althoughthis disclosure describes and illustrates particular components,devices, or systems carrying out particular steps of the method of FIG.30, this disclosure contemplates any suitable combination of anysuitable components, devices, or systems carrying out any suitable stepsof the method of FIG. 30.

FIG. 31 illustrates a social programming guide (SPG) 3110 which may bedisplayed on mobile device 840 or TV 830. In general, SPG 3110 providesan easy and convenient way for user 101 to view and select availablecontent to display on TV 830. Unlike a typical EPG that lists programsbased on channel numbers, SPG 3110 aggregates content from multiplesources (e.g., OTT providers 403, MSOs 402, on-demand programming, DVRs,etc.) into a convenient list of content according to the user'sinterests. The user is not required to know where the content is beingsourced from (e.g., from a STB or a DVR) or what channel is associatedwith the content. Instead, the user may simply tap an icon for anavailable program in SPG 3110 and the show will be displayed on TV 830.In certain embodiments, the available content in SPG 3110 is presentedin the form of graphics similar to apps on mobile device 840.

SPG 3110 may include any available content to be viewed on TV 830. Forexample, SPG 3110 may include content from MSOs 402 that would typicallybe selected using a STB such as STB/gateway 404 c. SPG 3110 may alsoinclude content from OTT provider 403 such as Hulu and Netflix. Contentin SPG 3110 may also be on-demand content and may include an indicationof the price to view the on-demand content. In some embodiments, SPG3110 may display a facepile proximate to available content. The facepilemay include, for example, images of other users (e.g., friends) who arewatching the content, who are planning to watch the content, who haveliked the content, or have interacted with the content in any other wayon social networking system 160.

In some embodiments, the content in SPG 3110 may include recommendedcontent for one or more users 101. For example, social networking system160 may determine who is in the room and then tailor the content in SPG3110 according to the determined user(s). For example, content in SPG3110 may include recommended content from playlist 2510 described above.

In some embodiments, SPG 3110 may be displayed on mobile device 840using, for example, a mobile app of social networking system 160. Insome embodiments, SPG 3110 may be displayed on TV 830. In embodimentswhere SPG 3110 is displayed on TV 830, it may overlay or cover aprogramming guide from MSOs 402. In certain embodiments, SPG 3110 maycompletely replace the user interface of a STB or DVR. In someembodiments, the overlay methods described above in reference to FIGS.8-12 may be utilized to hide the programming guide or channel numbersfrom MSO 402. As a result, users 101 may have an easier time locatingrelevant content to watch. In addition, there may be less confusionbecause users are not required to switch between multiple input sources.

In certain embodiments, sponsored shows may be included in SPG 3110. Forexample, a particular show or movie may be sponsored by MSO 402 or OTTprovider 403. An icon 3120 for the sponsored show may appear in SPG 3110where it may be more likely to be selected by the user. In someembodiments, the icon 3120 of the sponsored program may appearintermingled with recommended programs. In certain embodiments, asillustrated in FIG. 31, the icon 3120 of the sponsored program mayappear in a separate portion of SPG 3110 (e.g., at the bottom of thescreen of mobile device 840). In certain embodiments, MSO 402 or OTTprovider 403 may pay a fee for a sponsored program to be included in SPG3110.

In some embodiments, a particular icon in SPG 3110 may be a personalizedqueue for user 101. This personalize queue may include content that isthe most recommended content for user 101 and may dynamically update.For example, social networking system 160 may determine recommendationsfor user 101 using the methods described above in reference to FIGS.25-26. The recommendations may then be loaded into the personalizedqueue so that user 101 may simply push the icon of his personalizedqueue to being watching content recommended by social networking system160. The content in the personalized queue may be a playlist thatautomatically plays the content sequentially without the user having tointeract with mobile device 840, TV 830, or social TV dongle 810. Insome embodiments, an “ADD TO QUEUE” indication may be available to user101 in, for example, SPG 3110. This indication may allow user 101 toquickly place a program into his personalized queue. In someembodiments, when user selects an “ADD TO QUEUE” indication to add ShowA to his personalized queue, social networking system 160 mayautomatically post a message such as “User 101 is planning to watch ShowA” to social networking system 160. In some embodiments, friends 101 a-eof user 101 may suggest programs into the personalized queue of user101. This may be controlled by preferences of user 101.

In some embodiments, a particular icon in SPG 3110 may be arecommendation queue for user 101. This recommendation queue may includecontent that has been recommended to user 101 by friends 101 a-e. Forexample, a particular friend 101 a may want user 101 to view aparticular show. He may indicate this to social networking system 160using any appropriate button or link on social networking system 160.Social networking system 160 may then load the recommended show into therecommendation queue so that user 101 may simply push icon 3120 of hisrecommendation queue to being watching content recommended by hisfriends.

FIG. 32 illustrates an example method 3200 for providing a SPG. Themethod may begin at step 3210, where first available content from afirst content source is determined. For example, one or more TV showsmay be determined from a source such as MSO 402. At step 3220, firstavailable content from a second content source is determined. Forexample, one or more programs may be determined from a source such asOTT provider 403. In some embodiments, the first and second sources aredifferent sources. In certain embodiments, the content sources mayinclude a DVR, a STB, a website, a cable provider, an on-demandprovider, and the like.

At step 3230, recommended content for a first user is determined. Insome embodiments, the recommended content is determined using a socialgraph and the first and second available content. In some embodiments,social networking system 160 may query a social graph in order todetermine social coefficients for each of the first and second availablecontent. The social coefficients may be used in some embodiments todetermine the recommended content.

At step 3240, a programming guide is displayed on a display device ofthe first user. In some embodiments, the programming guide includes therecommended content for the first user. In some embodiments, theprogramming guide masks the first and second content sources. That is,some embodiments mask channel numbers of the recommended content. Insome embodiments, the programming guide is SPG 3110.

Particular embodiments may repeat one or more steps of the method ofFIG. 32, where appropriate. Although this disclosure describes andillustrates particular steps of the method of FIG. 32 as occurring in aparticular order, this disclosure contemplates any suitable steps of themethod of FIG. 32 occurring in any suitable order. Moreover, althoughthis disclosure describes and illustrates particular components,devices, or systems carrying out particular steps of the method of FIG.32, this disclosure contemplates any suitable combination of anysuitable components, devices, or systems carrying out any suitable stepsof the method of FIG. 32.

FIG. 33 illustrates a social graph 3300 that may be used to stream mediato any device of user 101. In some embodiments, social graph 3300 may bean embodiment of social graph 300 above. Social graph 3300 includesconcept nodes 304 for each device of user 101 that is capable ofpresenting any form of media (e.g., shows, movies, pictures, desktop,music, etc.) to user 101. For example, concept nodes 304 may beassociated with any mobile device 840 of user 101, a TV 380 of user 101,any user device 1710, any client system 130, or any other device of user101 capable of presenting media to user 101. In certain embodiments,concept nodes 304 of user device may be associated with an input or anoutput of the user device. Social graph 3300 also includes edges 306that may indicate that a user “owns” the user devices associated withconcept nodes 306.

In general, social graph 3300 with user device concept nodes 304 permitsa user to stream any media that friends or other connections of the userare consuming. In one example, a first user may begin viewing a show onhis TV at the same point in the show that one of his friends iscurrently viewing. As a result, the users may be able to simultaneouslyview the same TV show and interact with one another (e.g., chat asdiscussed above) through social networking system 160. As anotherexample, a user may be able to display photos from his mobile device onone or more of his friend's TVs or mobile devices. These any many otherexamples of streaming media from one user's device to another user'sdevice may be accomplished by social networking system 160 creating andanalyzing the nodes and edges of social graph 3300.

In some embodiments, social networking system 160 may determine whatservice or device a user is currently using in order to determine whatand how to stream media from another user. For example, socialnetworking system 160 may analyze a user's interactions with socialnetworking system 160 in order to determine whether the user iscurrently interacting with social networking system 160 using asmartphone, a computer, a tablet computer, or any other user device. Asanother example, social networking system 160 may determine whether auser is currently communicating with social networking system 160 usingWiFi, cellular communications such as 4G, or any other appropriatecommunications. Once the type of service or user device is determined,social networking system 160 may modify the content being shared fromanother user. For example, if social networking system 160 determinesthat a user is currently utilizing a smartphone over 4G, it may optimizea video stream to be streamed to the smartphone over 4G. As anotherexample, if social networking system 160 determines that a user iscurrently utilizing a TV 830 coupled to social TV dongle 810, social TVdongle 810 may optimize a video stream to be viewed on TV 830.

In some embodiments, social networking system 160 may determine anoptimal user device in which to display shared content from anotheruser. For example, social networking system 160 may determine that auser is attempting to view a video stream that a friend is currentlyviewing. Social networking system 160 may then determine that theoptimal display device for the shared video stream would be a TV 830. Asanother example, social networking system 160 may determine that a useris attempting to view a photo that a friend is currently viewing on hismobile device 840. Social networking system 160 may then determine thatthe optimal display device for the shared photo would be another mobiledevice 840.

FIG. 34 illustrates an example method 3400 for streaming media tomultiple user devices. The method may begin at step 3410, where a firstmedia device of a first user is determined. As step 3420, a second mediadevice of a second user is determined. For example, the media devicesmay include a smartphone, a TV, a digital music device, a laptop, atablet computer, and the like.

At step 3430, a first node for the first media device and a first edgebetween the first node and a node of the first user is created in asocial graph. In some embodiments, the social graph may refer to socialgraph 3300. In some embodiments, the first edge may be an “owns” edge306. In some embodiments, the first node corresponds to an input oroutput port of the first media device.

At step 3440, a second node for the second media device and a secondedge between the second node and a node of the second user is created inthe social graph. In some embodiments, the social graph may refer tosocial graph 3300. In some embodiments, the second edge may be an “owns”edge 306. In some embodiments, the second node corresponds to an inputor output port of the second media device.

At step 3450, media is streamed from the first media device to thesecond device using the first and second nodes and the first and secondedges created in step 3430 and 3440. For example, the first user mayview a TV show on his TV that the second user is currently viewing onhis TV. As another example, the first user may view photos from thesecond user's smartphone on his TV.

Particular embodiments may repeat one or more steps of the method ofFIG. 34, where appropriate. Although this disclosure describes andillustrates particular steps of the method of FIG. 34 as occurring in aparticular order, this disclosure contemplates any suitable steps of themethod of FIG. 34 occurring in any suitable order. Moreover, althoughthis disclosure describes and illustrates particular components,devices, or systems carrying out particular steps of the method of FIG.34, this disclosure contemplates any suitable combination of anysuitable components, devices, or systems carrying out any suitable stepsof the method of FIG. 34.

FIG. 35 illustrates a comment visualization graph for a particularprogram that may be generated and displayed to user 101. As discussedabove, comments, posts, chat messages, or any other content posted tosocial networking system 160 may be time-stamped with informationindicating when the content was posted. For example, a post about aparticular TV show may be time-stamped with a time that indicates whenin the show the post was made. As another example, chat messages about aparticular movie may be time-stamped to indicate at what point in themovie the chat messages were exchanged. Social networking system 160 mayanalyze content posted to social networking system 160 and determine allposts related to a particular program. A comment visualization graphsuch as the one illustrated in FIG. 35 may then be created and displayedto user 101 on, for example, mobile device 840 or TV 830. The commentvisualization graph presents a visual representation of the amount ofcomments made during the program and at what point during the programthey were made. As a result, users may be able to determine interestingor important parts of a program. In some embodiments, the users may thenbe able to jump that portion of the program and view the interesting orimportant content. A particular example of a comment visualization graphis discussed in more detail below.

The illustrated example of FIG. 35 is a comment visualization graph foran awards show such as the ACADEMY AWARDS. The x-axis in this embodimentis time relative to the beginning of the show and the y-axis is theamount of comments about the show. As illustrated in this figure,comments of users of social networking system 160 about the award showwere clumped in three main portions of the show: around 1:00, 10:00, and33:00. By presenting the amount of comments about the show relative totime, the comment visualization graph presents a convenient way forusers of social networking system 160 to determine important orinteresting portions of the show. In this example, users may deduce thatthe interesting or important parts of the show occurred 1:00, 10:00, and33:00 marks because that is when the most comments by user of socialnetworking system 160 were posted. In certain embodiments, the commentvisualization graph may also present a thumbnail image of the show atthe important or interesting points of the show (e.g., when the mostcomments occurred). In this example, thumbnail images of the show at the1:00, 10:00, and 33:00 marks are displayed above the clumps of comments.This may allow the user to quickly discern what the comments wereregarding. In some embodiments, the user may click or otherwise interactwith the thumbnail images in order to being playing the show at thatpoint.

In certain embodiments, a dynamic highlight reel may be presented to auser based on which sections of a program received the highest densityof user comments. For example, a user may interact with mobileapplication 602 on his mobile device and indicate that he would like towatch the top highlights of a certain program in a certain amount oftime. An example might include, “I want to watch the top highlights ofthe ACADEMY AWARDS in three minutes.” One or more of mobile application602, social TV dongle 810, and social networking system 160 may thenanalyze content posted to social networking system 160 (e.g.,time-stamped user comments about the ACADEMY AWARDS) and determine allposts related to the ACADEMY AWARDS. Clips of portions of the ACADEMYAWARDS having the most user comments may then be automaticallyassembled. In certain embodiments, the clips may be ranked from thehighest density of user comments to the lowest. In some embodiments, asubset of the determined clips is chosen so that the chosen clips willrun in the amount of time indicated by the user. A highlight reel of thedetermined clips may then be presented to the user on any appropriatedisplay device.

In some embodiments, an index of scenes or interesting parts of aprogram may be presented to user 101. For example, the comments for acertain program may be analyzed as described above in order to determineimportant or interesting portions of the program. The determinedimportant or interesting portions of the program may then be ranked orotherwise listed in an index and presented to user 101. In certainembodiments, for example, the index may be presented to user 101interposed onto, for example, the user's time line.

In some embodiments, content and interactions other than user commentsmay be analyzed and presented to user 101 in a visualization graphsimilar to the comment visualization graph of FIG. 35 above. Forexample, time-stamped likes of a particular program may be analyzed andgraphed to show the various points within the program that usersselected to like the program. In another example, observed dataindicating interest in the content may be analyzed and graphed to showthe various points within the program that users were intensely watchingthe content. As a particular example, users' eye gaze, as describedabove, may be analyzed and graphed according to time stamps to showwhich portions of a show users were transfixed on the TV.

FIG. 36 illustrates an example method 3600 for providing an examplecomment visualization graph. The method may begin at step 3610, where aplurality of user comments related to a particular program aredetermined. In some embodiments, a social graph such as social graph 300may be utilized to determine the plurality of comments. For example,social networking system 160 may query social graph 300 for all commentsrelated to an awards show that previously aired.

At step 3620, a time associated with each comment as related to theprogram is determined. In some embodiments, each comment may betime-stamped with an absolute time or a time relative to the program.For example, a comment may be time-stamped with a time indicating thatit was posted at 33:00 into the show.

At step 3630, a comment visualization graph is presented. In someembodiments, the comment visualization graph shows an indication of theamount of comments plotted against time. For example, the commentvisualization graph may plot time of the program on the x-axis and thenumber of comments made about the program on the y-axis. This may allowusers to quickly discern important parts of the program. For example, ifthere were many comments surrounding the time of 33:00 into the program,the user may determine that the 33:00 mark was an important part of theprogram. In some embodiments, the program may automatically beginplaying the important parts of the program.

Particular embodiments may repeat one or more steps of the method ofFIG. 36, where appropriate. Although this disclosure describes andillustrates particular steps of the method of FIG. 36 as occurring in aparticular order, this disclosure contemplates any suitable steps of themethod of FIG. 36 occurring in any suitable order. Moreover, althoughthis disclosure describes and illustrates particular components,devices, or systems carrying out particular steps of the method of FIG.36, this disclosure contemplates any suitable combination of anysuitable components, devices, or systems carrying out any suitable stepsof the method of FIG. 36.

In some embodiments, social networking system 160 may provideuser-submitted content about viewed programs to entities external tosocial networking system 160. For example, social networking system 160may (for a fee) provide user comments about a particular TV show to theproviders or producers of the show, subject to user settings. As anotherexample, social networking system 160 may (for a fee) provide the numberof times on any screen (e.g., any mobile device, TV, or computer) that auser has seen an advertisement from a specific media campaign, subjectto user settings. In some embodiments, the user comments may be providedin real-time or at a time after the airing of the show. In anotherexample, social networking system 160 may (for a fee) provide user likesof a product, user posts about a product, or the number of times on anyscreen (e.g., any mobile device, TV, or computer) that a user has seenan advertisement for a particular product. Both of these examples arediscussed in more detail below.

In some embodiments, social networking system 160 may provideuser-generated content about a viewed program to the program's provider.For example, consider a scenario where a pilot of a TV show is viewed byusers of social networking system 160. Either while viewing the pilot orat a time after the airing of the pilot, users of social networkingsystem 160 may submit posts, comments, chat sessions, or any othercontent regarding the pilot. Social networking system 160 may aggregatethe user-submitted content for the pilot and provide it to the pilot'screator or producer either while the pilot is airing or after it hasaired. For example, social networking system 160 may present on a mobiledevice 840 or TV 830 of the pilot's producer the comments that users ofsocial networking system 160 are making about the pilot in real-time. Asa result, content providers may be able to obtain viewer feedback in amuch more timely manner.

In some embodiments, social networking system 160 may provideuser-generated content about a product to the product's advertiser ormaker, subject to user privacy settings. For example, consider ascenario where an advertisement for a product is aired and then viewedby users of social networking system 160. Either while viewing theadvertisement or at a time after the airing of the advertisement, usersof social networking system 160 may submit posts, comments, chatsessions, or any other content regarding the product in theadvertisement. Social networking system 160 may aggregate theuser-submitted content for the product and provide it to the advertiseror maker of the product.

In some embodiments, social networking system 160 may rate movies,advertisements, shows, episodes of shows, seasons of shows, or any othergranularity of shows based on metrics of the viewers of the show,subject to user privacy settings. For example, social networking system160 may rate a show based on how many users 101 watched the show. Thismay be determined, for example, using the methods of determiningidentities of one or more users 101 who are watching content on TV 830described in reference to FIGS. 15-16 above. In some embodiments, socialnetworking system 160 may rate a show based on other metrics such as howmany users 101 finished a show, how many users 101 continue to watch newepisodes of a show, how many users post content to social networkingsystem 160 about the show, or take any other social action with socialnetworking system 160 regarding the show. In some embodiments, socialnetworking system 160 may provide the determined ratings to contentproviders so that the content providers may refine their shows. Forexample, the determined ratings may allow the content provider to removea character if social networking system 160 determines that usersstopped watching new episodes of the show when the character wasintroduced.

In some embodiments, social networking system 160 may provide the abovedetermined ratings of advertisements to advertisers or makers of theproduct being advertised. For example, social networking system 160 mayrate an advertisement based on metrics such as how many users 101 postcontent to social networking system 160 about the advertisement or takeany other social action with social networking system 160 regarding theshow (e.g., how many users have liked the product). In some embodiments,social networking system 160 may provide the determined ratings toadvertisers or makers of the product for a fee.

In some embodiments, the determined ratings may be utilized by socialnetworking system 160 to provide recommendations to users 101. Forexample, SPG 3110 and playlist 2510 may include shows that have beendetermined to have high ratings. As another example, notifications maybe sent to users 101 to view shows that have been determined to havehigh ratings.

In some embodiments, analytics may be performed by social networkingsystem 160 to determine metrics regarding the size and composition ofaudiences for movies and TV shows. For example, as described above,social networking system 160 may determine what content users 101 ofsocial networking system 160 are watching. Social networking system 160may aggregate this data in order to determine the total number of users101 across social networking system 160 that viewed a particularprogram. In some embodiments, social networking system 160 may alsodetermine metrics regarding the composition of the viewers of theparticular program. For example, social networking system 160 mayanalyze user data of the users 101 determined to have viewed aparticular program in order to determine the age, gender, etc. of thedetermined viewers. Social networking system 160 may then provide themetrics regarding the viewing audience of a particular show to contentproviders, advertisers, or any other appropriate entity.

FIG. 37A illustrates an example personal computing device 3700. Inparticular embodiments, personal computing device 3700 may comprise aprocessor 3710, a memory 3720, a communication component 3730 (e.g.,antenna and communication interface for wireless communications), one ormore input and/or output (I/O) components and/or interfaces 3740, andone or more sensors 3750. In particular embodiments, one or more I/Ocomponents and/or interfaces 3740 may incorporate one or more sensors3750. In particular embodiments, personal computing device 3700 maycomprise a computer system or and element thereof as described in FIG.39 and associated description.

In particular embodiments, a personal computing device, such as a mobiledevice, may include various types of sensors 3750, such as, for exampleand without limitation: touch sensors (disposed, for example, on adisplay of the device, the back of the device and/or one or more lateraledges of the device) for detecting a user touching the surface of themobile electronic device (e.g., using one or more fingers);accelerometer for detecting whether the personal computing device 3700is moving and the speed of the movement; thermometer for measuring thetemperature change near the personal computing device 3700; proximitysensor for detecting the proximity of the personal computing device 3700to another object (e.g., a hand, desk, or other object); light sensorfor measuring the ambient light around the personal computing device3700; imaging sensor (e.g., camera) for capturing digital still imagesand/or video of objects near the personal computing device 3700 (e.g.,scenes, people, bar codes, QR codes, etc.); location sensors (e.g.,Global Positioning System (GPS)) for determining the location (e.g., interms of latitude and longitude) of the mobile electronic device;sensors for detecting communication networks within close proximity(e.g., near field communication (NFC), Bluetooth, RFID, infrared);chemical sensors; biometric sensors for biometrics-based (e.g.,fingerprint, palm vein pattern, hand geometry, iris/retina, DNA, face,voice, olfactory, sweat) authentication of user of personal computingdevice 3700; etc. This disclosure contemplates that a mobile electronicdevice may include any applicable type of sensor. Sensors may providevarious types of sensor data, which may be analyzed to determine theuser's intention with respect to the mobile electronic device at a giventime.

In particular embodiments, a sensors hub 3760 may optionally be includedin personal computing device 3700. Sensors 3750 may be connected tosensors hub 3760, which may be a low power-consuming processor thatcontrols sensors 3750, manages power for sensors 3750, processes sensorinputs, aggregates sensor data, and performs certain sensor functions.In addition, in particular embodiments, some types of sensors 3750 maybe connected to a controller 3770. In this case, sensors hub 3760 may beconnected to controller 3770, which in turn is connected to sensor 3750.Alternatively, in particular embodiments, there may be a sensor monitorin place of sensors hub 3760 for managing sensors 3750.

In particular embodiments, in addition to the front side, personalcomputing device 3700 may have one or more sensors for performingbiometric identification. Such sensors may be positioned on any surfaceof personal computing device 3700. In example embodiments, as the user'shand touches personal computing device 3700 to grab hold of it, thetouch sensors may capture the user's fingerprints or palm vein pattern.In example embodiments, while a user is viewing the screen of personalcomputing device 3700, a camera may capture an image of the user's faceto perform facial recognition. In example embodiments, while a user isviewing the screen of personal computing device 3700, an infraredscanner may scan the user's iris and/or retina. In example embodiments,while a user is in contact or close proximity with personal computingdevice 3700, chemical and/or olfactory sensors may capture relevant dataabout a user. In particular embodiments, upon detecting that there is achange in state with respect to the identity of the user utilizingpersonal computing device 3700, either by itself or in combination withother types of sensor indications, personal computing device 3700 maydetermine that it is being shared.

In particular embodiments, in addition to the front side, the personalcomputing device 3700 may have touch sensors on the left and rightsides. Optionally, the personal computing device 3700 may also havetouch sensors on the back, top, or bottom side. Thus, as the user's handtouches personal computing device 3700 to grab hold of it, the touchsensors may detect the user's fingers or palm touching personalcomputing device 3700. In particular embodiments, upon detecting thatthere is a change in state with respect to a user touching personalcomputing device 3700, either by itself or in combination with othertypes of sensor indications, personal computing device 3700 maydetermine that it is being shared.

In particular embodiments, personal computing device 3700 may have anaccelerometer in addition to or instead of the touch sensors on the leftand right sides. Sensor data provided by the accelerometer may also beused to estimate whether a new user has picked up personal computingdevice 3700 from a resting position, e.g., on a table or desk, displayshelf, or from someone's hand or from within someone's bag. When theuser picks up personal computing device 3700 and brings it in front ofthe user's face, there may be a relatively sudden increase in themovement speed of personal computing device 3700. This change in thedevice's movement speed may be detected based on the sensor datasupplied by the accelerometer. In particular embodiments, upon detectingthat there is a significant increase in the speed of the device'smovement, either by itself or in combination with other types of sensorindications, personal computing device 3700 may determine that it isbeing shared.

In particular embodiments, personal computing device 3700 may have aGyrometer in addition or instead of the touch sensors on the left andright sides. A Gyrometer, also known as a gyroscope, is a device formeasuring the orientation along one or more axis. In particularembodiments, a Gyrometer may be used to measure the orientation ofpersonal computing device 3700. When personal computing device 3700 isstored on a shelf or in the user's bag, it may stay mostly in oneorientation. However, when the user grabs hold of personal computingdevice 3700 and lifts it up and/or moves it closer to bring it in frontof the user's face, there may be a relatively sudden change in theorientation of personal computing device 3700. The orientation ofpersonal computing device 3700 may be detected and measured by thegyrometer. If the orientation of personal computing device 3700 haschanged significantly, In particular embodiments, upon detecting thatthere is a significant change in the orientation of personal computingdevice 3700, either by itself or in combination with other types ofsensor indications, personal computing device 3700 may determine that itis being shared.

In particular embodiments, personal computing device 3700 may have alight sensor. When personal computing device 3700 is stored in a user'spocket or case, it is relatively dark around personal computing device3700. On the other hand, when the user brings personal computing device3700 out of his pocket, it may be relatively bright around personalcomputing device 3700, especially during day time or in well-lit areas.The sensor data supplied by the light sensor may be analyzed to detectwhen a significant change in the ambient light level around personalcomputing device 3700 occurs. In particular embodiments, upon detectingthat there is a significant increase in the ambient light level aroundpersonal computing device 3700, either by itself or in combination withother types of sensor indications, personal computing device 3700 maydetermine that it is being shared.

In particular embodiments, personal computing device 3700 may have aproximity sensor. The sensor data supplied by the proximity sensor maybe analyzed to detect when personal computing device 3700 is in closeproximity to a specific object, such as the user's hand. For example,mobile device 3700 may have an infrared LED (light-emitting diode) 3790(i.e., proximity sensor) placed on its back side. When the user holdssuch a mobile device in his hand, the palm of the user's hand may coverinfrared LED 3790. As a result, infrared LED 3790 may detect when theuser's hand is in close proximity to mobile device 3700. In particularembodiments, upon detecting that personal computing device 3700 is inclose proximity to the user's hand, either by itself or in combinationwith other types of sensor indications, personal computing device 3700may determine that it is being shared.

A personal computing device 3700 may have any number of sensors ofvarious types, and these sensors may supply different types of sensordata. Different combinations of the individual types of sensor data maybe used together to detect and estimate a user's current intention withrespect to personal computing device 3700 (e.g., whether the user reallymeans to take personal computing device 3700 out of his pocket and useit). Sometimes, using multiple types of sensor data in combination mayyield a more accurate, and thus better, estimation of the user'sintention with respect to personal computing device 3700 at a given timethan only using a single type of sensor data. Nevertheless, it ispossible to estimate the user's intention using a single type of sensordata (e.g., touch-sensor data).

FIG. 37B illustrates the exterior of an example personal computingdevice 3700. Personal computing device 3700 has approximately six sides:front, back, top, bottom, left, and right. Touch sensors may be placedanywhere on any of the six sides of personal computing device 3700. Forexample, in FIG. 37B, a touchscreen incorporating touch sensors 3780A isplaced on the front of personal computing device 3700. The touchscreenmay function as an input/output (I/O) component for personal computingdevice 3700. In addition, touch sensors 3780B and 3780C are placed onthe left and right sides of personal computing device 3700,respectively. Touch sensors 3780B and 3780C may detect a user's handtouching the sides of personal computing device 3700. In particularembodiments, touch sensors 3780A, 3780B, 3780C may be implemented usingresistive, capacitive, and/or inductive touch sensors. The electrodes ofthe touch sensors 3780A, 3780B, 3780C may be arranged on a thin solidpiece of material or a thin wire mesh. In the case of capacitive touchsensors, there may be two types of electrodes: transmitting andreceiving. These electrodes may be connected to a controller (e.g.,controller 3770 illustrated in FIG. 3A), which may be a microchipdesigned to drive the transmitting electrodes with electrical pulses andmeasure the changes in capacitance from the receiving electrodes causedby a user's touches in order to detect the locations of the usertouches.

Of course, personal computing device 3700 is merely an example. Inpractice, a device may have any number of sides, and this disclosurecontemplates devices with any number of sides. The touch sensors may beplaced on any side of a device.

In particular embodiments, personal computing device 3700 may have aproximity sensor 3790 (e.g., an infrared LED) placed on its back side.Proximity sensor 3790 may be able to supply sensor data for determiningits proximity, and thus the proximity of personal computing device 3700,to another object.

FIG. 38 illustrates an example software architecture 3800 forinformation and applications on a personal computing device 3700. Inparticular embodiments, software architecture 3800 may comprise software3810 and data store(s) 3820. In particular embodiments, personalinformation may be stored in an application data cache 3820 and/or aprofile data store 3820 and/or another data store 3820. In particularembodiments, one or more software applications may be executed onpersonal computing device 3700. In particular embodiments, they may beweb-based applications hosted on servers. For example, a web-basedapplication may be associated with a URI (Uniform Resource Identifier)or URL (Uniform Resource Locator). From personal computing device 3700,a user may access the web-based application through its associated URIor URL (e.g., by using a web browser). Alternatively, in otherembodiments, they may be native applications installed and residing onpersonal computing device 3700. Thus, software 3810 may also include anynumber of application user interfaces 3830 and application functions3840. For example, one application (e.g., Google Maps®) may enable adevice user to view a map, search for addresses and businesses, and getdirections; a second application may enable the device user to read,send, and receive emails; a third application (e.g., a web browser) mayenable the device user to browse and search the Internet; a fourthapplication may enable the device user to take photos or record videosusing personal computing device 3700; a fifth application may allow thedevice user to receive and initiate VoIP and/or cellular network calls,and so on. Each application has one or more specific functionalities,and the software (e.g., one or more software modules) implementing thesefunctionalities may be included in application functions 3840. Eachapplication may also have a user interface that enables the device userto interact with the application, and the software implementing theapplication user interface may be included in application userinterfaces 3830. In particular embodiments, the functionalities of anapplication may be implemented using JavaScript®, Java®, C, or othersuitable programming languages. In particular embodiments, the userinterface of an application may be implemented using HyperText MarkupLanguage (HTML), JavaScript®, Java®, or other suitable programminglanguages.

In particular embodiments, the user interface of an application mayinclude any number of screens or displays. In particular embodiments,each screen or display of the user interface may be implemented as a webpage. Thus, the device user may interact with the application through aseries of screens or displays (i.e., a series of web pages). Inparticular embodiments, operating system 3850 is Google's Android™mobile technology platform. With Android®, there is a Java® packagecalled “android.webkit”, which provides various tools for browsing theweb. Among the “android.webkit” package, there is a Java class called“android.webkit.WebView”, which implements a View for displaying webpages. This class uses the WebKit rendering engine to display web pagesand includes methods to navigate forward and backward through a history,zoom in, zoom out, perform text searches, and so on. In particularembodiments, an application user interface 3830 may utilize Android'sWebView API to display each web page of the user interface in a Viewimplemented by the “android.webkit.WebView” class. Thus, in particularembodiments, software 3810 may include any number of web views 3860,each for displaying one or more web pages that implement the userinterface of an application.

During the execution of an application, the device user may interactwith the application through its user interface. For example, the usermay provide inputs to the application in various displays (e.g., webpages). Outputs of the application may be presented to the user invarious displays (e.g., web pages) as well. In particular embodiments,when the user provides an input to the application through a specificdisplay (e.g., a specific web page), an event (e.g., an input event) maybe generated by, for example, a web view 3860 or application userinterfaces 3830. Each input event may be forwarded to applicationfunctions 3840, or application functions 3840 may listen for inputevents thus generated. When application functions 3840 receive an inputevent, the appropriate software module in application functions 3840 maybe invoked to process the event. In addition, specific functionalitiesprovided by operating system 3850 and/or hardware (e.g., as described inFIGS. 3A-B) may also be invoked. For example, if the event is generatedas a result of the user pushing a button to take a photo with personalcomputing device 3700, a corresponding image processing module may beinvoked to convert the raw image data into an image file (e.g., JPG orGIF) and store the image file in the storage 3820 of personal computingdevice 3700. As anther example, if the event is generated as a result ofthe user selecting an icon to compose an instant message, thecorresponding short message service (SMS) module may be invoked toenable the user to compose and send the message.

In particular embodiments, when an output of the application is ready tobe presented to the user, an event (e.g., an output event) may begenerated by, for example, a software module in application functions3840 or operating system 3850. Each output event may be forwarded toapplication user interfaces 3830, or application user interfaces 3830may listen for output events thus generated. When application userinterfaces 3830 receive an output event, it may construct a web view3860 to display a web page representing or containing the output. Forexample, in response to the user selecting an icon to compose an instantmessage, an output may be constructed that includes a text field thatallows the user to input the message. This output may be presented tothe user as a web page and displayed to the user in a web view 3860 sothat the user may type into the text field the message to be sent.

The user interface of an application may be implemented using a suitableprogramming language (e.g., HTML, JavaScript®, or Java®). Morespecifically, in particular embodiments, each web page that implements ascreen or display of the user interface may be implemented using asuitable programming language. In particular embodiments, when a webview 3860 is constructed to display a web page (e.g., by applicationuser interfaces 3830 in response to an output event), the codeimplementing the web page is loaded into web view 3860.

FIG. 39 illustrates an example computer system 3900. In particularembodiments, one or more computer systems 3900 perform one or more stepsof one or more methods described or illustrated herein. In particularembodiments, one or more computer systems 3900 provide functionalitydescribed or illustrated herein. In particular embodiments, softwarerunning on one or more computer systems 3900 performs one or more stepsof one or more methods described or illustrated herein or providesfunctionality described or illustrated herein. Particular embodimentsinclude one or more portions of one or more computer systems 3900.Herein, reference to a computer system may encompass a computing device,where appropriate. Moreover, reference to a computer system mayencompass one or more computer systems, where appropriate.

This disclosure contemplates any suitable number of computer systems3900. This disclosure contemplates computer system 3900 taking anysuitable physical form. As example and not by way of limitation,computer system 3900 may be an embedded computer system, asystem-on-chip (SOC), a single-board computer system (SBC) (such as, forexample, a computer-on-module (COM) or system-on-module (SOM)), adesktop computer system, a laptop or notebook computer system, aninteractive kiosk, a mainframe, a mesh of computer systems, a mobiletelephone, a personal digital assistant (PDA), a server, a tabletcomputer system, or a combination of two or more of these. Whereappropriate, computer system 3900 may include one or more computersystems 3900; be unitary or distributed; span multiple locations; spanmultiple machines; span multiple data centers; or reside in a cloud,which may include one or more cloud components in one or more networks.Where appropriate, one or more computer systems 3900 may perform withoutsubstantial spatial or temporal limitation one or more steps of one ormore methods described or illustrated herein. As an example and not byway of limitation, one or more computer systems 3900 may perform in realtime or in batch mode one or more steps of one or more methods describedor illustrated herein. One or more computer systems 3900 may perform atdifferent times or at different locations one or more steps of one ormore methods described or illustrated herein, where appropriate.

In particular embodiments, computer system 3900 includes a processor3902, memory 3904, storage 3906, an input/output (I/O) interface 3908, acommunication interface 3910, and a bus 3912. Although this disclosuredescribes and illustrates a particular computer system having aparticular number of particular components in a particular arrangement,this disclosure contemplates any suitable computer system having anysuitable number of any suitable components in any suitable arrangement.

In particular embodiments, processor 3902 includes hardware forexecuting instructions, such as those making up a computer program. Asan example and not by way of limitation, to execute instructions,processor 3902 may retrieve (or fetch) the instructions from an internalregister, an internal cache, memory 3904, or storage 3906; decode andexecute them; and then write one or more results to an internalregister, an internal cache, memory 3904, or storage 3906. In particularembodiments, processor 3902 may include one or more internal caches fordata, instructions, or addresses. This disclosure contemplates processor3902 including any suitable number of any suitable internal caches,where appropriate. As an example and not by way of limitation, processor3902 may include one or more instruction caches, one or more datacaches, and one or more translation lookaside buffers (TLBs).Instructions in the instruction caches may be copies of instructions inmemory 3904 or storage 3906, and the instruction caches may speed upretrieval of those instructions by processor 3902. Data in the datacaches may be copies of data in memory 3904 or storage 3906 forinstructions executing at processor 3902 to operate on; the results ofprevious instructions executed at processor 3902 for access bysubsequent instructions executing at processor 3902 or for writing tomemory 3904 or storage 3906; or other suitable data. The data caches mayspeed up read or write operations by processor 3902. The TLBs may speedup virtual-address translation for processor 3902. In particularembodiments, processor 3902 may include one or more internal registersfor data, instructions, or addresses. This disclosure contemplatesprocessor 3902 including any suitable number of any suitable internalregisters, where appropriate. Where appropriate, processor 3902 mayinclude one or more arithmetic logic units (ALUs); be a multi-coreprocessor; or include one or more processors 3902. Although thisdisclosure describes and illustrates a particular processor, thisdisclosure contemplates any suitable processor.

In particular embodiments, memory 3904 includes main memory for storinginstructions for processor 3902 to execute or data for processor 3902 tooperate on. As an example and not by way of limitation, computer system3900 may load instructions from storage 3906 or another source (such as,for example, another computer system 3900) to memory 3904. Processor3902 may then load the instructions from memory 3904 to an internalregister or internal cache. To execute the instructions, processor 3902may retrieve the instructions from the internal register or internalcache and decode them. During or after execution of the instructions,processor 3902 may write one or more results (which may be intermediateor final results) to the internal register or internal cache. Processor3902 may then write one or more of those results to memory 3904. Inparticular embodiments, processor 3902 executes only instructions in oneor more internal registers or internal caches or in memory 3904 (asopposed to storage 3906 or elsewhere) and operates only on data in oneor more internal registers or internal caches or in memory 3904 (asopposed to storage 3906 or elsewhere). One or more memory buses (whichmay each include an address bus and a data bus) may couple processor3902 to memory 3904. Bus 3912 may include one or more memory buses, asdescribed below. In particular embodiments, one or more memorymanagement units (MMUs) reside between processor 3902 and memory 3904and facilitate accesses to memory 3904 requested by processor 3902. Inparticular embodiments, memory 3904 includes random access memory (RAM).This RAM may be volatile memory, where appropriate. Where appropriate,this RAM may be dynamic RAM (DRAM) or static RAM (SRAM). Moreover, whereappropriate, this RAM may be single-ported or multi-ported RAM. Thisdisclosure contemplates any suitable RAM. Memory 3904 may include one ormore memories 3904, where appropriate. Although this disclosuredescribes and illustrates particular memory, this disclosurecontemplates any suitable memory.

In particular embodiments, storage 3906 includes mass storage for dataor instructions. As an example and not by way of limitation, storage3906 may include a hard disk drive (HDD), a floppy disk drive, flashmemory, an optical disc, a magneto-optical disc, magnetic tape, or aUniversal Serial Bus (USB) drive or a combination of two or more ofthese. Storage 3906 may include removable or non-removable (or fixed)media, where appropriate. Storage 3906 may be internal or external tocomputer system 3900, where appropriate. In particular embodiments,storage 3906 is non-volatile, solid-state memory. In particularembodiments, storage 3906 includes read-only memory (ROM). Whereappropriate, this ROM may be mask-programmed ROM, programmable ROM(PROM), erasable PROM (EPROM), electrically erasable PROM (EEPROM),electrically alterable ROM (EAROM), or flash memory or a combination oftwo or more of these. This disclosure contemplates mass storage 3906taking any suitable physical form. Storage 3906 may include one or morestorage control units facilitating communication between processor 3902and storage 3906, where appropriate. Where appropriate, storage 3906 mayinclude one or more storages 3906. Although this disclosure describesand illustrates particular storage, this disclosure contemplates anysuitable storage.

In particular embodiments, I/O interface 3908 includes hardware,software, or both providing one or more interfaces for communicationbetween computer system 3900 and one or more I/O devices. Computersystem 3900 may include one or more of these I/O devices, whereappropriate. One or more of these I/O devices may enable communicationbetween a person and computer system 3900. As an example and not by wayof limitation, an I/O device may include a keyboard, keypad, microphone,monitor, mouse, printer, scanner, speaker, still camera, stylus, tablet,touch screen, trackball, video camera, another suitable I/O device or acombination of two or more of these. An I/O device may include one ormore sensors. This disclosure contemplates any suitable I/O devices andany suitable I/O interfaces 3908 for them. Where appropriate, I/Ointerface 3908 may include one or more device or software driversenabling processor 3902 to drive one or more of these I/O devices. I/Ointerface 3908 may include one or more I/O interfaces 3908, whereappropriate. Although this disclosure describes and illustrates aparticular I/O interface, this disclosure contemplates any suitable I/Ointerface.

In particular embodiments, communication interface 3910 includeshardware, software, or both providing one or more interfaces forcommunication (such as, for example, packet-based communication) betweencomputer system 3900 and one or more other computer systems 3900 or oneor more networks. As an example and not by way of limitation,communication interface 3910 may include a network interface controller(NIC) or network adapter for communicating with an Ethernet or otherwire-based network or a wireless NIC (WNIC) or wireless adapter forcommunicating with a wireless network, such as a WI-FI network. Thisdisclosure contemplates any suitable network and any suitablecommunication interface 3910 for it. As an example and not by way oflimitation, computer system 3900 may communicate with an ad hoc network,a personal area network (PAN), a local area network (LAN), a wide areanetwork (WAN), a metropolitan area network (MAN), or one or moreportions of the Internet or a combination of two or more of these. Oneor more portions of one or more of these networks may be wired orwireless. As an example, computer system 3900 may communicate with awireless PAN (WPAN) (such as, for example, a BLUETOOTH WPAN), a WI-FInetwork, a WI-MAX network, a cellular telephone network (such as, forexample, a Global System for Mobile Communications (GSM) network), orother suitable wireless network or a combination of two or more ofthese. Computer system 3900 may include any suitable communicationinterface 3910 for any of these networks, where appropriate.Communication interface 3910 may include one or more communicationinterfaces 3910, where appropriate. Although this disclosure describesand illustrates a particular communication interface, this disclosurecontemplates any suitable communication interface.

In particular embodiments, bus 3912 includes hardware, software, or bothcoupling components of computer system 3900 to each other. As an exampleand not by way of limitation, bus 3912 may include an AcceleratedGraphics Port (AGP) or other graphics bus, an Enhanced Industry StandardArchitecture (EISA) bus, a front-side bus (FSB), a HYPERTRANSPORT (HT)interconnect, an Industry Standard Architecture (ISA) bus, an INFINIBANDinterconnect, a low-pin-count (LPC) bus, a memory bus, a Micro ChannelArchitecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, aPCI-Express (PCIe) bus, a serial advanced technology attachment (SATA)bus, a Video Electronics Standards Association local (VLB) bus, oranother suitable bus or a combination of two or more of these. Bus 3912may include one or more buses 3912, where appropriate. Although thisdisclosure describes and illustrates a particular bus, this disclosurecontemplates any suitable bus or interconnect.

Herein, a computer-readable non-transitory storage medium or media mayinclude one or more semiconductor-based or other integrated circuits(ICs) (such, as for example, field-programmable gate arrays (FPGAs) orapplication-specific ICs (ASICs)), hard disk drives (HDDs), hybrid harddrives (HHDs), optical discs, optical disc drives (ODDs),magneto-optical discs, magneto-optical drives, floppy diskettes, floppydisk drives (FDDs), magnetic tapes, solid-state drives (SSDs),RAM-drives, SECURE DIGITAL cards or drives, any other suitablecomputer-readable non-transitory storage media, or any suitablecombination of two or more of these, where appropriate. Acomputer-readable non-transitory storage medium may be volatile,non-volatile, or a combination of volatile and non-volatile, whereappropriate.

Herein, “or” is inclusive and not exclusive, unless expressly indicatedotherwise or indicated otherwise by context. Therefore, herein, “A or B”means “A, B, or both,” unless expressly indicated otherwise or indicatedotherwise by context. Moreover, “and” is both joint and several, unlessexpressly indicated otherwise or indicated otherwise by context.Therefore, herein, “A and B” means “A and B, jointly or severally,”unless expressly indicated otherwise or indicated otherwise by context.

The scope of this disclosure encompasses all changes, substitutions,variations, alterations, and modifications to the example embodimentsdescribed or illustrated herein that a person having ordinary skill inthe art would comprehend. The scope of this disclosure is not limited tothe example embodiments described or illustrated herein. Moreover,although this disclosure describes and illustrates respectiveembodiments herein as including particular components, elements,functions, operations, or steps, any of these embodiments may includeany combination or permutation of any of the components, elements,functions, operations, or steps described or illustrated anywhere hereinthat a person having ordinary skill in the art would comprehend.Furthermore, reference in the appended claims to an apparatus or systemor a component of an apparatus or system being adapted to, arranged to,capable of, configured to, enabled to, operable to, or operative toperform a particular function encompasses that apparatus, system,component, whether or not it or that particular function is activated,turned on, or unlocked, as long as that apparatus, system, or componentis so adapted, arranged, capable, configured, enabled, operable, oroperative.

What is claimed is:
 1. A method comprising: by one or more computersystems of a social-networking system, querying a social graph of thesocial-networking system for social content associated with particularmedia content, the social graph comprising a plurality of nodes andedges connecting the nodes, the nodes comprising user nodes that areeach associated with a particular user of the social-networking system;by the one or more computer systems of the social-networking system,determining a time associated with each of the social content; by theone or more computer systems of the social-networking system,determining, using the determined time of each of the social content, aparticular point within the particular media content that the socialcontent was generated; by the one or more computer systems of thesocial-networking system, presenting a content visualization graph fordisplay on a display device, the content visualization graph comprisinga visual indication of the particular points within the particular mediacontent that the social content was generated; determining a pluralityof interesting points of the particular media content based on an amountof social content generated, wherein determining the plurality ofinteresting points comprises: determining a respective amount of socialcontent generated for each of a plurality of time periods; ranking theplurality of time periods based on the respective amount of socialcontent generated for each of the plurality of time periods; andselecting, as the plurality of interesting points, a pre-determinednumber of the plurality of time periods that have the largest amount ofgenerated social content according to the ranking; displaying aplurality of thumbnail images on the content visualization graph, eachof the thumbnail images depicting a portion of one of the interestingpoints of the particular media content, each of the thumbnail imagesbeing located on the content visualization graph proximate to itsassociated interesting point of the particular media content; by the oneor more computer systems of the social-networking system, receiving userinput that indicates an amount of time in which to view highlights ofthe particular media content; and by the one or more computer systems ofthe social-networking system, generating a highlight reel comprising atleast some of the interesting points of the particular media content,the highlight reel generated to play within the amount of time indicatedby the user input.
 2. The method of claim 1, wherein the social contentassociated with the particular media content comprises one or more of: acomment; a post; a message; an action corresponding to watching theparticular media content; and “liking” the particular media content. 3.The method of claim 1, wherein each thumbnail image is operable, whenselected, to begin playing the particular media content at the point ofthe particular media content indicated by the thumbnail image.
 4. Themethod of claim 1, further comprising, by the one or more computersystems of the social-networking system: ranking the determinedinteresting points of the particular media content, wherein theinteresting points of the highlight reel comprise the ranked interestingpoints.
 5. The method of claim 1, wherein the content visualizationgraph comprises: an x-axis comprising a time relative to the particularmedia content; and a y-axis comprising an amount of the social contentassociated with the particular media content.
 6. The method of claim 1,wherein the social content associated with the particular media contentis time-stamped.
 7. One or more computer-readable non-transitory storagemedia in one or more computing systems, the media embodying logic thatis operable when executed to: query a social graph of asocial-networking system for social content associated with particularmedia content, the social graph comprising a plurality of nodes andedges connecting the nodes, the nodes comprising user nodes that areeach associated with a particular user of the social-networking system;determine a time associated with each of the social content; determine,using the determined time of each of the social content, a particularpoint within the particular media content that the social content wasgenerated; present a content visualization graph for display on adisplay device, the content visualization graph comprising a visualindication of the particular points within the particular media contentthat the social content was generated; determine a plurality ofinteresting points of the particular media content based on an amount ofsocial content generated, wherein determining the plurality ofinteresting points comprises: determining a respective amount of socialcontent generated for each of a plurality of time periods; ranking theplurality of time periods based on the respective amount of socialcontent generated for each of the plurality of time periods; andselecting, as the plurality of interesting points, a pre-determinednumber of the plurality of time periods that have the largest amount ofgenerated social content according to the ranking; display a pluralityof thumbnail images on the content visualization graph, each of thethumbnail images depicting a portion of one of the interesting points ofthe particular media content, each of the thumbnail images being locatedon the content visualization graph proximate to its associatedinteresting point of the particular media content; receive user inputthat indicates an amount of time in which to view highlights of theparticular media content; and generate a highlight reel comprising atleast some of the interesting points of the particular media content,the highlight reel generated to play within the amount of time indicatedby the user input.
 8. The media of claim 7, wherein the social contentassociated with the particular media content comprises one or more of: acomment; a post; a message; an action corresponding to watching theparticular media content; and “liking” the particular media content. 9.The media of claim 7, wherein each thumbnail image is operable, whenselected, to begin playing the particular media content at the point ofthe particular media content indicated by the thumbnail image.
 10. Themedia of claim 7, the logic further operable to: rank the determinedinteresting points of the particular media content, wherein theinteresting points of the highlight reel comprise the ranked interestingpoints.
 11. The media of claim 7, wherein the content visualizationgraph comprises: an x-axis comprising a time relative to the particularmedia content; and a y-axis comprising an amount of the social contentassociated with the particular media content.
 12. The media of claim 7,wherein the social content associated with the particular media contentis time-stamped.
 13. A system, comprising: one or more memory devices;and a processor communicatively coupled to the one or more memorydevices, the processor operable to: query a social graph of asocial-networking system for social content associated with particularmedia content, the social graph comprising a plurality of nodes andedges connecting the nodes, the nodes comprising user nodes that areeach associated with a particular user of the social-networking system;determine a time associated with each of the social content; determine,using the determined time of each of the social content, a particularpoint within the particular media content that the social content wasgenerated; present a content visualization graph for display on adisplay device, the content visualization graph comprising a visualindication of the particular points within the particular media contentthat the social content was generated; determine a plurality ofinteresting points of the particular media content based on an amount ofsocial content generated, wherein determining the plurality ofinteresting points comprises: determining a respective amount of socialcontent generated for each of a plurality of time periods; ranking theplurality of time periods based on the respective amount of socialcontent generated for each of the plurality of time periods; andselecting, as the plurality of interesting points, a pre-determinednumber of the plurality of time periods that have the largest amount ofgenerated social content according to the ranking; display a pluralityof thumbnail images on the content visualization graph, each of thethumbnail images depicting a portion of one of the interesting points ofthe particular media content, each of the thumbnail images being locatedon the content visualization graph proximate to its associatedinteresting point of the particular media content; receive user inputthat indicates an amount of time in which to view highlights of theparticular media content; and generate a highlight reel comprising atleast some of the interesting points of the particular media content,the highlight reel generated to play within the amount of time indicatedby the user input.
 14. The system of claim 13, wherein the socialcontent associated with the particular media content comprises one ormore of: a comment; a post; a message; an action corresponding towatching the particular media content; and “liking” the particular mediacontent.
 15. The system of claim 13, wherein each thumbnail image isoperable, when selected, to begin playing the particular media contentat the point of the particular media content indicated by the thumbnailimage.
 16. The system of claim 13, the processor further operable to:rank the determined interesting points of the particular media content,wherein the interesting points of the highlight reel comprise the rankedinteresting points.
 17. The system of claim 13, wherein the contentvisualization graph comprises: an x-axis comprising a time relative tothe particular media content; and a y-axis comprising an amount of thesocial content associated with the particular media content.